<!DOCTYPE html>
<html class="client-nojs vector-feature-night-mode-disabled vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-1 vector-feature-appearance-pinned-clientpref-1 vector-sticky-header-enabled" lang="en" dir="ltr"><head>
<meta charset="UTF-8">
<title>Word embedding</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="canonical" href="https://en.wikipedia.org/wiki/Word_embedding"> <link href="./mw/ext.cite.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.icons.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.search.codex.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/user.styles.css" rel="stylesheet" type="text/css">
<meta name="ResourceLoaderDynamicStyles" content="">
<link rel="stylesheet" type="text/css" href="./mw/site.styles.css">
<link rel="stylesheet" type="text/css" href="./mw/noscript.css">
<link rel="stylesheet" type="text/css" href="./footer.css">
<link rel="stylesheet" type="text/css" href="./vector-2022.css">
</head>
<body class="skin--responsive skin-vector skin-vector-search-vue mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Word_embedding rootpage-Word_embedding skin-vector-2022 action-view">
<div class="mw-page-container">
<div class="mw-page-container-inner">
<div class="mw-content-container">
<main id="content" class="mw-body">
<header class="mw-body-header vector-page-titlebar">
<h1 id="firstHeading" class="firstHeading mw-first-heading">
<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Word embedding</span></span>
</h1>
</header>
<a id="top"></a>
<div id="bodyContent" class="vector-body ve-init-mw-desktopArticleTarget-targetContainer" aria-labelledby="firstHeading" data-mw-ve-target-container="">
<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr">
<style data-mw-deduplicate="TemplateStyles:r1129693374">
/* start https://en.wikipedia.org/ */
.mw-parser-output .hlist dl,.mw-parser-output .hlist ol,.mw-parser-output .hlist ul{margin:0;padding:0}.mw-parser-output .hlist dd,.mw-parser-output .hlist dt,.mw-parser-output .hlist li{margin:0;display:inline}.mw-parser-output .hlist.inline,.mw-parser-output .hlist.inline dl,.mw-parser-output .hlist.inline ol,.mw-parser-output .hlist.inline ul,.mw-parser-output .hlist dl dl,.mw-parser-output .hlist dl ol,.mw-parser-output .hlist dl ul,.mw-parser-output .hlist ol dl,.mw-parser-output .hlist ol ol,.mw-parser-output .hlist ol ul,.mw-parser-output .hlist ul dl,.mw-parser-output .hlist ul ol,.mw-parser-output .hlist ul ul{display:inline}.mw-parser-output .hlist .mw-empty-li{display:none}.mw-parser-output .hlist dt::after{content:": "}.mw-parser-output .hlist dd::after,.mw-parser-output .hlist li::after{content:" · ";font-weight:bold}.mw-parser-output .hlist dd:last-child::after,.mw-parser-output .hlist dt:last-child::after,.mw-parser-output .hlist li:last-child::after{content:none}.mw-parser-output .hlist dd dd:first-child::before,.mw-parser-output .hlist dd dt:first-child::before,.mw-parser-output .hlist dd li:first-child::before,.mw-parser-output .hlist dt dd:first-child::before,.mw-parser-output .hlist dt dt:first-child::before,.mw-parser-output .hlist dt li:first-child::before,.mw-parser-output .hlist li dd:first-child::before,.mw-parser-output .hlist li dt:first-child::before,.mw-parser-output .hlist li li:first-child::before{content:" (";font-weight:normal}.mw-parser-output .hlist dd dd:last-child::after,.mw-parser-output .hlist dd dt:last-child::after,.mw-parser-output .hlist dd li:last-child::after,.mw-parser-output .hlist dt dd:last-child::after,.mw-parser-output .hlist dt dt:last-child::after,.mw-parser-output .hlist dt li:last-child::after,.mw-parser-output .hlist li dd:last-child::after,.mw-parser-output .hlist li dt:last-child::after,.mw-parser-output .hlist li li:last-child::after{content:")";font-weight:normal}.mw-parser-output .hlist ol{counter-reset:listitem}.mw-parser-output .hlist ol>li{counter-increment:listitem}.mw-parser-output .hlist ol>li::before{content:" "counter(listitem)"\a0 "}.mw-parser-output .hlist dd ol>li:first-child::before,.mw-parser-output .hlist dt ol>li:first-child::before,.mw-parser-output .hlist li ol>li:first-child::before{content:" ("counter(listitem)"\a0 "}
/* end https://en.wikipedia.org/ */
</style><style data-mw-deduplicate="TemplateStyles:r1246091330">
/* start https://en.wikipedia.org/ */
.mw-parser-output .sidebar{width:22em;float:right;clear:right;margin:0.5em 0 1em 1em;background:var(--background-color-neutral-subtle,#f8f9fa);border:1px solid var(--border-color-base,#a2a9b1);padding:0.2em;text-align:center;line-height:1.4em;font-size:88%;border-collapse:collapse;display:table}body.skin-minerva .mw-parser-output .sidebar{display:table!important;float:right!important;margin:0.5em 0 1em 1em!important}.mw-parser-output .sidebar-subgroup{width:100%;margin:0;border-spacing:0}.mw-parser-output .sidebar-left{float:left;clear:left;margin:0.5em 1em 1em 0}.mw-parser-output .sidebar-none{float:none;clear:both;margin:0.5em 1em 1em 0}.mw-parser-output .sidebar-outer-title{padding:0 0.4em 0.2em;font-size:125%;line-height:1.2em;font-weight:bold}.mw-parser-output .sidebar-top-image{padding:0.4em}.mw-parser-output .sidebar-top-caption,.mw-parser-output .sidebar-pretitle-with-top-image,.mw-parser-output .sidebar-caption{padding:0.2em 0.4em 0;line-height:1.2em}.mw-parser-output .sidebar-pretitle{padding:0.4em 0.4em 0;line-height:1.2em}.mw-parser-output .sidebar-title,.mw-parser-output .sidebar-title-with-pretitle{padding:0.2em 0.8em;font-size:145%;line-height:1.2em}.mw-parser-output .sidebar-title-with-pretitle{padding:0.1em 0.4em}.mw-parser-output .sidebar-image{padding:0.2em 0.4em 0.4em}.mw-parser-output .sidebar-heading{padding:0.1em 0.4em}.mw-parser-output .sidebar-content{padding:0 0.5em 0.4em}.mw-parser-output .sidebar-content-with-subgroup{padding:0.1em 0.4em 0.2em}.mw-parser-output .sidebar-above,.mw-parser-output .sidebar-below{padding:0.3em 0.8em;font-weight:bold}.mw-parser-output .sidebar-collapse .sidebar-above,.mw-parser-output .sidebar-collapse .sidebar-below{border-top:1px solid #aaa;border-bottom:1px solid #aaa}.mw-parser-output .sidebar-navbar{text-align:right;font-size:115%;padding:0 0.4em 0.4em}.mw-parser-output .sidebar-list-title{padding:0 0.4em;text-align:left;font-weight:bold;line-height:1.6em;font-size:105%}.mw-parser-output .sidebar-list-title-c{padding:0 0.4em;text-align:center;margin:0 3.3em}@media(max-width:640px){body.mediawiki .mw-parser-output .sidebar{width:100%!important;clear:both;float:none!important;margin-left:0!important;margin-right:0!important}}body.skin--responsive .mw-parser-output .sidebar a>img{max-width:none!important}@media screen{html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-list-title,html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle{background:transparent!important}html.skin-theme-clientpref-night .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle a{color:var(--color-progressive)!important}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-list-title,html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle{background:transparent!important}html.skin-theme-clientpref-os .mw-parser-output .sidebar:not(.notheme) .sidebar-title-with-pretitle a{color:var(--color-progressive)!important}}@media print{body.ns-0 .mw-parser-output .sidebar{display:none!important}}
/* end https://en.wikipedia.org/ */
</style><style data-mw-deduplicate="TemplateStyles:r886047488">
/* start https://en.wikipedia.org/ */
.mw-parser-output .nobold{font-weight:normal}
/* end https://en.wikipedia.org/ */
</style><table class="sidebar sidebar-collapse nomobile nowraplinks"><tbody><tr><td class="sidebar-pretitle">Part of a series on</td></tr><tr><th class="sidebar-title-with-pretitle"><a href="Machine_learning" title="Machine learning">Machine learning</a><br>and <a href="Data_mining" title="Data mining">data mining</a></th></tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Paradigms</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Supervised_learning" title="Supervised learning">Supervised learning</a></li>
<li><a href="Unsupervised_learning" title="Unsupervised learning">Unsupervised learning</a></li>
<li><a href="Semi-supervised_learning" class="mw-redirect" title="Semi-supervised learning">Semi-supervised learning</a></li>
<li><a href="Self-supervised_learning" title="Self-supervised learning">Self-supervised learning</a></li>
<li><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a></li>
<li><a href="Meta-learning_(computer_science)" title="Meta-learning (computer science)">Meta-learning</a></li>
<li><a href="Online_machine_learning" title="Online machine learning">Online learning</a></li>
<li><a href="Batch_learning" class="mw-redirect" title="Batch learning">Batch learning</a></li>
<li><a href="Curriculum_learning" title="Curriculum learning">Curriculum learning</a></li>
<li><a href="Rule-based_machine_learning" title="Rule-based machine learning">Rule-based learning</a></li>
<li><a href="Neuro-symbolic_AI" title="Neuro-symbolic AI">Neuro-symbolic AI</a></li>
<li><a href="Neuromorphic_engineering" class="mw-redirect" title="Neuromorphic engineering">Neuromorphic engineering</a></li>
<li><a href="Quantum_machine_learning" title="Quantum machine learning">Quantum machine learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Problems</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Statistical_classification" title="Statistical classification">Classification</a></li>
<li><a href="Generative_model" title="Generative model">Generative modeling</a></li>
<li><a href="Regression_analysis" title="Regression analysis">Regression</a></li>
<li><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></li>
<li><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></li>
<li><a href="Density_estimation" title="Density estimation">Density estimation</a></li>
<li><a href="Anomaly_detection" title="Anomaly detection">Anomaly detection</a></li>
<li><a href="Data_cleaning" class="mw-redirect" title="Data cleaning">Data cleaning</a></li>
<li><a href="Automated_machine_learning" title="Automated machine learning">AutoML</a></li>
<li><a href="Association_rule_learning" title="Association rule learning">Association rules</a></li>
<li><a href="Semantic_analysis_(machine_learning)" title="Semantic analysis (machine learning)">Semantic analysis</a></li>
<li><a href="Structured_prediction" title="Structured prediction">Structured prediction</a></li>
<li><a href="Feature_engineering" title="Feature engineering">Feature engineering</a></li>
<li><a href="Feature_learning" title="Feature learning">Feature learning</a></li>
<li><a href="Learning_to_rank" title="Learning to rank">Learning to rank</a></li>
<li><a href="Grammar_induction" title="Grammar induction">Grammar induction</a></li>
<li><a href="Ontology_learning" title="Ontology learning">Ontology learning</a></li>
<li><a href="Multimodal_learning" title="Multimodal learning">Multimodal learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><div style="display: inline-block; line-height: 1.2em; padding: .1em 0;"><a href="Supervised_learning" title="Supervised learning">Supervised learning</a><br><span class="nobold"><span style="font-size: 85%;">(<b><a href="Statistical_classification" title="Statistical classification">classification</a></b> • <b><a href="Regression_analysis" title="Regression analysis">regression</a></b>)</span></span> </div></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Apprenticeship_learning" title="Apprenticeship learning">Apprenticeship learning</a></li>
<li><a href="Decision_tree_learning" title="Decision tree learning">Decision trees</a></li>
<li><a href="Ensemble_learning" title="Ensemble learning">Ensembles</a>
<ul><li><a href="Bootstrap_aggregating" title="Bootstrap aggregating">Bagging</a></li>
<li><a href="Boosting_(machine_learning)" title="Boosting (machine learning)">Boosting</a></li>
<li><a href="Random_forest" title="Random forest">Random forest</a></li></ul></li>
<li><a href="K-nearest_neighbors_algorithm" title="K-nearest neighbors algorithm"><i>k</i>-NN</a></li>
<li><a href="Linear_regression" title="Linear regression">Linear regression</a></li>
<li><a href="Naive_Bayes_classifier" title="Naive Bayes classifier">Naive Bayes</a></li>
<li><a href="Artificial_neural_network" class="mw-redirect" title="Artificial neural network">Artificial neural networks</a></li>
<li><a href="Logistic_regression" title="Logistic regression">Logistic regression</a></li>
<li><a href="Perceptron" title="Perceptron">Perceptron</a></li>
<li><a href="Relevance_vector_machine" title="Relevance vector machine">Relevance vector machine (RVM)</a></li>
<li><a href="Support_vector_machine" title="Support vector machine">Support vector machine (SVM)</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="BIRCH" title="BIRCH">BIRCH</a></li>
<li><a href="CURE_algorithm" title="CURE algorithm">CURE</a></li>
<li><a href="Hierarchical_clustering" title="Hierarchical clustering">Hierarchical</a></li>
<li><a href="K-means_clustering" title="K-means clustering"><i>k</i>-means</a></li>
<li><a href="Fuzzy_clustering" title="Fuzzy clustering">Fuzzy</a></li>
<li><a href="Expectation%E2%80%93maximization_algorithm" title="Expectation–maximization algorithm">Expectation–maximization (EM)</a></li>
<li><br><a href="DBSCAN" title="DBSCAN">DBSCAN</a></li>
<li><a href="OPTICS_algorithm" title="OPTICS algorithm">OPTICS</a></li>
<li><a href="Mean_shift" title="Mean shift">Mean shift</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Factor_analysis" title="Factor analysis">Factor analysis</a></li>
<li><a href="Canonical_correlation" title="Canonical correlation">CCA</a></li>
<li><a href="Independent_component_analysis" title="Independent component analysis">ICA</a></li>
<li><a href="Linear_discriminant_analysis" title="Linear discriminant analysis">LDA</a></li>
<li><a href="Non-negative_matrix_factorization" title="Non-negative matrix factorization">NMF</a></li>
<li><a href="Principal_component_analysis" title="Principal component analysis">PCA</a></li>
<li><a href="Proper_generalized_decomposition" title="Proper generalized decomposition">PGD</a></li>
<li><a href="T-distributed_stochastic_neighbor_embedding" title="T-distributed stochastic neighbor embedding">t-SNE</a></li>
<li><a href="Sparse_dictionary_learning" title="Sparse dictionary learning">SDL</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Structured_prediction" title="Structured prediction">Structured prediction</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Graphical_model" title="Graphical model">Graphical models</a>
<ul><li><a href="Bayesian_network" title="Bayesian network">Bayes net</a></li>
<li><a href="Conditional_random_field" title="Conditional random field">Conditional random field</a></li>
<li><a href="Hidden_Markov_model" title="Hidden Markov model">Hidden Markov</a></li></ul></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Anomaly_detection" title="Anomaly detection">Anomaly detection</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Random_sample_consensus" title="Random sample consensus">RANSAC</a></li>
<li><a href="K-nearest_neighbors_algorithm" title="K-nearest neighbors algorithm"><i>k</i>-NN</a></li>
<li><a href="Local_outlier_factor" title="Local outlier factor">Local outlier factor</a></li>
<li><a href="Isolation_forest" title="Isolation forest">Isolation forest</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Neural_network_(machine_learning)" title="Neural network (machine learning)">Neural networks</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Autoencoder" title="Autoencoder">Autoencoder</a></li>
<li><a href="Deep_learning" title="Deep learning">Deep learning</a></li>
<li><a href="Feedforward_neural_network" title="Feedforward neural network">Feedforward neural network</a></li>
<li><a href="Recurrent_neural_network" title="Recurrent neural network">Recurrent neural network</a>
<ul><li><a href="Long_short-term_memory" title="Long short-term memory">LSTM</a></li>
<li><a href="Gated_recurrent_unit" title="Gated recurrent unit">GRU</a></li>
<li><a href="Echo_state_network" title="Echo state network">ESN</a></li>
<li><a href="Reservoir_computing" title="Reservoir computing">reservoir computing</a></li></ul></li>
<li><a href="Boltzmann_machine" title="Boltzmann machine">Boltzmann machine</a>
<ul><li><a href="Restricted_Boltzmann_machine" title="Restricted Boltzmann machine">Restricted</a></li></ul></li>
<li><a href="Generative_adversarial_network" title="Generative adversarial network">GAN</a></li>
<li><a href="Diffusion_model" title="Diffusion model">Diffusion model</a></li>
<li><a href="Self-organizing_map" title="Self-organizing map">SOM</a></li>
<li><a href="Convolutional_neural_network" title="Convolutional neural network">Convolutional neural network</a>
<ul><li><a href="U-Net" title="U-Net">U-Net</a></li>
<li><a href="LeNet" title="LeNet">LeNet</a></li>
<li><a href="AlexNet" title="AlexNet">AlexNet</a></li>
<li><a href="DeepDream" title="DeepDream">DeepDream</a></li></ul></li>
<li><a href="Neural_field" title="Neural field">Neural field</a>
<ul><li><a href="Neural_radiance_field" title="Neural radiance field">Neural radiance field</a></li>
<li><a href="Physics-informed_neural_networks" title="Physics-informed neural networks">Physics-informed neural networks</a></li></ul></li>
<li><a href="Transformer_(deep_learning_architecture)" title="Transformer (deep learning architecture)">Transformer</a>
<ul><li><a href="Vision_transformer" title="Vision transformer">Vision</a></li></ul></li>
<li><a href="Mamba_(deep_learning_architecture)" title="Mamba (deep learning architecture)">Mamba</a></li>
<li><a href="Spiking_neural_network" title="Spiking neural network">Spiking neural network</a></li>
<li><a href="Memtransistor" title="Memtransistor">Memtransistor</a></li>
<li><a href="Electrochemical_RAM" title="Electrochemical RAM">Electrochemical RAM</a> (ECRAM)</li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)"><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a></div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Q-learning" title="Q-learning">Q-learning</a></li>
<li><a href="Policy_gradient_method" title="Policy gradient method">Policy gradient</a></li>
<li><a href="State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action" title="State–action–reward–state–action">SARSA</a></li>
<li><a href="Temporal_difference_learning" title="Temporal difference learning">Temporal difference (TD)</a></li>
<li><a href="Multi-agent_reinforcement_learning" title="Multi-agent reinforcement learning">Multi-agent</a>
<ul><li><a href="Self-play_(reinforcement_learning_technique)" class="mw-redirect" title="Self-play (reinforcement learning technique)">Self-play</a></li></ul></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Learning with humans</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Active_learning_(machine_learning)" title="Active learning (machine learning)">Active learning</a></li>
<li><a href="Crowdsourcing" title="Crowdsourcing">Crowdsourcing</a></li>
<li><a href="Human-in-the-loop" title="Human-in-the-loop">Human-in-the-loop</a></li>
<li><a href="Mechanistic_interpretability" title="Mechanistic interpretability">Mechanistic interpretability</a></li>
<li><a href="Reinforcement_learning_from_human_feedback" title="Reinforcement learning from human feedback">RLHF</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Model diagnostics</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Coefficient_of_determination" title="Coefficient of determination">Coefficient of determination</a></li>
<li><a href="Confusion_matrix" title="Confusion matrix">Confusion matrix</a></li>
<li><a href="Learning_curve_(machine_learning)" title="Learning curve (machine learning)">Learning curve</a></li>
<li><a href="Receiver_operating_characteristic" title="Receiver operating characteristic">ROC curve</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Mathematical foundations</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Kernel_machines" class="mw-redirect" title="Kernel machines">Kernel machines</a></li>
<li><a href="Bias%E2%80%93variance_tradeoff" title="Bias–variance tradeoff">Bias–variance tradeoff</a></li>
<li><a href="Computational_learning_theory" title="Computational learning theory">Computational learning theory</a></li>
<li><a href="Empirical_risk_minimization" title="Empirical risk minimization">Empirical risk minimization</a></li>
<li><a href="Occam_learning" title="Occam learning">Occam learning</a></li>
<li><a href="Probably_approximately_correct_learning" title="Probably approximately correct learning">PAC learning</a></li>
<li><a href="Statistical_learning_theory" title="Statistical learning theory">Statistical learning</a></li>
<li><a href="Vapnik%E2%80%93Chervonenkis_theory" title="Vapnik–Chervonenkis theory">VC theory</a></li>
<li><a href="Topological_deep_learning" title="Topological deep learning">Topological deep learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Journals and conferences</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="AAAI_Conference_on_Artificial_Intelligence" title="AAAI Conference on Artificial Intelligence">AAAI</a></li>
<li><a href="ECML_PKDD" title="ECML PKDD">ECML PKDD</a></li>
<li><a href="Conference_on_Neural_Information_Processing_Systems" title="Conference on Neural Information Processing Systems">NeurIPS</a></li>
<li><a href="International_Conference_on_Machine_Learning" title="International Conference on Machine Learning">ICML</a></li>
<li><a href="International_Conference_on_Learning_Representations" title="International Conference on Learning Representations">ICLR</a></li>
<li><a href="International_Joint_Conference_on_Artificial_Intelligence" title="International Joint Conference on Artificial Intelligence">IJCAI</a></li>
<li><a href="Machine_Learning_(journal)" title="Machine Learning (journal)">ML</a></li>
<li><a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">JMLR</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-content">
<div class="sidebar-list mw-collapsible mw-collapsed machine-learning-list-title"><div class="sidebar-list-title" style="border-top:1px solid #aaa; text-align:center;;color: var(--color-base)">Related articles</div><div class="sidebar-list-content mw-collapsible-content hlist">
<ul><li><a href="Glossary_of_artificial_intelligence" title="Glossary of artificial intelligence">Glossary of artificial intelligence</a></li>
<li><a href="List_of_datasets_for_machine-learning_research" title="List of datasets for machine-learning research">List of datasets for machine-learning research</a>
<ul><li><a href="List_of_datasets_in_computer_vision_and_image_processing" title="List of datasets in computer vision and image processing">List of datasets in computer vision and image processing</a></li></ul></li>
<li><a href="Outline_of_machine_learning" title="Outline of machine learning">Outline of machine learning</a></li></ul></div></div></td>
</tr><tr><td class="sidebar-navbar"><style data-mw-deduplicate="TemplateStyles:r1239400231">
/* start https://en.wikipedia.org/ */
.mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:"[ "}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:" ]"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}html.skin-theme-clientpref-night .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}@media(prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .navbar li a abbr{color:var(--color-base)!important}}@media print{.mw-parser-output .navbar{display:none!important}}
/* end https://en.wikipedia.org/ */
</style></td></tr></tbody></table>
<p>In <a href="Natural_language_processing" title="Natural language processing">natural language processing</a>, a <b>word embedding</b> is a representation of a word. The <a href="Embedding_(machine_learning)" title="Embedding (machine learning)">embedding</a> is used in <a href="Content_analysis" title="Content analysis">text analysis</a>. Typically, the representation is a <a href="Real_number" title="Real number">real-valued</a> vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning.<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup> Word embeddings can be obtained using <a href="Language_model" title="Language model">language modeling</a> and <a href="Feature_learning" title="Feature learning">feature learning</a> techniques, where words or phrases from the vocabulary are mapped to <a href="Vector_(mathematics)" class="mw-redirect" title="Vector (mathematics)">vectors</a> of <a href="Real_numbers" class="mw-redirect" title="Real numbers">real numbers</a>.
</p><p>Methods to generate this mapping include <a href="Neural_net_language_model" class="mw-redirect" title="Neural net language model">neural networks</a>,<sup id="cite_ref-2" class="reference"><a href="#cite_note-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup> <a href="Dimensionality_reduction" title="Dimensionality reduction">dimensionality reduction</a> on the word <a href="Co-occurrence_matrix" title="Co-occurrence matrix">co-occurrence matrix</a>,<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-4" class="reference"><a href="#cite_note-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup> probabilistic models,<sup id="cite_ref-6" class="reference"><a href="#cite_note-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup> explainable knowledge base method,<sup id="cite_ref-7" class="reference"><a href="#cite_note-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup> and explicit representation in terms of the context in which words appear.<sup id="cite_ref-8" class="reference"><a href="#cite_note-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup>
</p><p>Word and phrase embeddings, when used as the underlying input representation, have been shown to boost the performance in NLP tasks such as <a href="Syntactic_parsing" class="mw-redirect" title="Syntactic parsing">syntactic parsing</a><sup id="cite_ref-9" class="reference"><a href="#cite_note-9"><span class="cite-bracket">[</span>9<span class="cite-bracket">]</span></a></sup> and <a href="Sentiment_analysis" title="Sentiment analysis">sentiment analysis</a>.<sup id="cite_ref-10" class="reference"><a href="#cite_note-10"><span class="cite-bracket">[</span>10<span class="cite-bracket">]</span></a></sup>
</p>
<meta property="mw:PageProp/toc">
<div class="mw-heading mw-heading2"><h2 id="Development_and_history_of_the_approach">Development and history of the approach</h2></div>
<p>In <a href="Distributional_semantics" title="Distributional semantics">distributional semantics</a>, a quantitative methodological approach for understanding meaning in observed language, word embeddings or semantic <a href="Feature_space" class="mw-redirect" title="Feature space">feature space</a> models have been used as a knowledge representation for some time.<sup id="cite_ref-11" class="reference"><a href="#cite_note-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup> Such models aim to quantify and categorize semantic similarities between linguistic items based on their distributional properties in large samples of language data. The underlying idea that "a word is characterized by the company it keeps" was proposed in a 1957 article by <a href="J._R._Firth" class="mw-redirect" title="J. R. Firth">John Rupert Firth</a>,<sup id="cite_ref-12" class="reference"><a href="#cite_note-12"><span class="cite-bracket">[</span>12<span class="cite-bracket">]</span></a></sup> but also has roots in the contemporaneous work on search systems<sup id="cite_ref-13" class="reference"><a href="#cite_note-13"><span class="cite-bracket">[</span>13<span class="cite-bracket">]</span></a></sup> and in cognitive psychology.<sup id="cite_ref-14" class="reference"><a href="#cite_note-14"><span class="cite-bracket">[</span>14<span class="cite-bracket">]</span></a></sup>
</p><p>The notion of a semantic space with lexical items (words or multi-word terms) represented as vectors or embeddings is based on the computational challenges of capturing distributional characteristics and using them for practical application to measure similarity between words, phrases, or entire documents. The first generation of semantic space models is the <a href="Vector_space_model" title="Vector space model">vector space model</a> for information retrieval.<sup id="cite_ref-Salton_original_15-0" class="reference"><a href="#cite_note-Salton_original-15"><span class="cite-bracket">[</span>15<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-SaltonEA_CACM_16-0" class="reference"><a href="#cite_note-SaltonEA_CACM-16"><span class="cite-bracket">[</span>16<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-17" class="reference"><a href="#cite_note-17"><span class="cite-bracket">[</span>17<span class="cite-bracket">]</span></a></sup> Such vector space models for words and their distributional data implemented in their simplest form results in a very sparse vector space of high dimensionality (cf. <a href="Curse_of_dimensionality" title="Curse of dimensionality">curse of dimensionality</a>). Reducing the number of dimensions using linear algebraic methods such as <a href="Singular-value_decomposition" class="mw-redirect" title="Singular-value decomposition">singular value decomposition</a> then led to the introduction of <a href="Latent_semantic_analysis" title="Latent semantic analysis">latent semantic analysis</a> in the late 1980s and the <a href="Random_indexing" title="Random indexing">random indexing</a> approach for collecting word co-occurrence contexts.<sup id="cite_ref-18" class="reference"><a href="#cite_note-18"><span class="cite-bracket">[</span>18<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-19" class="reference"><a href="#cite_note-19"><span class="cite-bracket">[</span>19<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-20" class="reference"><a href="#cite_note-20"><span class="cite-bracket">[</span>20<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-21" class="reference"><a href="#cite_note-21"><span class="cite-bracket">[</span>21<span class="cite-bracket">]</span></a></sup> In 2000, <a href="Yoshua_Bengio" title="Yoshua Bengio">Bengio</a> et al. provided in a series of papers titled "Neural probabilistic language models" to reduce the high dimensionality of word representations in contexts by "learning a distributed representation for words".<sup id="cite_ref-22" class="reference"><a href="#cite_note-22"><span class="cite-bracket">[</span>22<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-23" class="reference"><a href="#cite_note-23"><span class="cite-bracket">[</span>23<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-24" class="reference"><a href="#cite_note-24"><span class="cite-bracket">[</span>24<span class="cite-bracket">]</span></a></sup>
</p><p>A study published in <a href="NeurIPS" class="mw-redirect" title="NeurIPS">NeurIPS</a> (NIPS) 2002 introduced the use of both word and document embeddings applying the method of kernel CCA to bilingual (and multi-lingual) corpora, also providing an early example of <a href="Self-supervised_learning" title="Self-supervised learning">self-supervised learning</a> of word embeddings.<sup id="cite_ref-25" class="reference"><a href="#cite_note-25"><span class="cite-bracket">[</span>25<span class="cite-bracket">]</span></a></sup>
</p><p>Word embeddings come in two different styles, one in which words are expressed as vectors of co-occurring words, and another in which words are expressed as vectors of linguistic contexts in which the words occur; these different styles are studied in Lavelli et al., 2004.<sup id="cite_ref-26" class="reference"><a href="#cite_note-26"><span class="cite-bracket">[</span>26<span class="cite-bracket">]</span></a></sup> Roweis and Saul published in <i><a href="Science_(journal)" title="Science (journal)">Science</a></i> how to use "<a href="Nonlinear_dimensionality_reduction#Locally-linear_embedding" title="Nonlinear dimensionality reduction">locally linear embedding</a>" (LLE) to discover representations of high dimensional data structures.<sup id="cite_ref-27" class="reference"><a href="#cite_note-27"><span class="cite-bracket">[</span>27<span class="cite-bracket">]</span></a></sup> Most new word embedding techniques after about 2005 rely on a <a href="Neural_network" title="Neural network">neural network</a> architecture instead of more probabilistic and algebraic models, after foundational work done by Yoshua Bengio<sup id="cite_ref-28" class="reference"><a href="#cite_note-28"><span class="cite-bracket">[</span>28<span class="cite-bracket">]</span></a></sup> and colleagues.<sup id="cite_ref-29" class="reference"><a href="#cite_note-29"><span class="cite-bracket">[</span>29<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-30" class="reference"><a href="#cite_note-30"><span class="cite-bracket">[</span>30<span class="cite-bracket">]</span></a></sup>
</p><p>The approach has been adopted by many research groups after theoretical advances in 2010 had been made on the quality of vectors and the training speed of the model, as well as after hardware advances allowed for a broader <a href="Parameter_space" title="Parameter space">parameter space</a> to be explored profitably. In 2013, a team at <a href="Google" title="Google">Google</a> led by <a href="Tomas_Mikolov" class="mw-redirect" title="Tomas Mikolov">Tomas Mikolov</a> created <a href="Word2vec" title="Word2vec">word2vec</a>, a word embedding toolkit that can train vector space models faster than previous approaches. The word2vec approach has been widely used in experimentation and was instrumental in raising interest for word embeddings as a technology, moving the research strand out of specialised research into broader experimentation and eventually paving the way for practical application.<sup id="cite_ref-31" class="reference"><a href="#cite_note-31"><span class="cite-bracket">[</span>31<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading2"><h2 id="Polysemy_and_homonymy">Polysemy and homonymy</h2></div>
<p>Historically, one of the main limitations of static word embeddings or word <a href="Vector_space_model" title="Vector space model">vector space models</a> is that words with multiple meanings are conflated into a single representation (a single vector in the semantic space). In other words, <a href="Polysemy" title="Polysemy">polysemy</a> and <a href="Homonym" title="Homonym">homonymy</a> are not handled properly. For example, in the sentence "The club I tried yesterday was great!", it is not clear if the term <i>club</i> is related to the word sense of a <i><a href="Club_sandwich" title="Club sandwich">club sandwich</a></i>, <i><a href="Meeting_house" title="Meeting house">clubhouse</a></i>, <i><a href="Golf_club" title="Golf club">golf club</a></i>, or any other sense that <i>club</i> might have. The necessity to accommodate multiple meanings per word in different vectors (multi-sense embeddings) is the motivation for several contributions in NLP to split single-sense embeddings into multi-sense ones.<sup id="cite_ref-32" class="reference"><a href="#cite_note-32"><span class="cite-bracket">[</span>32<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-33" class="reference"><a href="#cite_note-33"><span class="cite-bracket">[</span>33<span class="cite-bracket">]</span></a></sup>
</p><p>Most approaches that produce multi-sense embeddings can be divided into two main categories for their word sense representation, i.e., unsupervised and knowledge-based.<sup id="cite_ref-34" class="reference"><a href="#cite_note-34"><span class="cite-bracket">[</span>34<span class="cite-bracket">]</span></a></sup> Based on <a href="Word2vec" title="Word2vec">word2vec</a> skip-gram, Multi-Sense Skip-Gram (MSSG)<sup id="cite_ref-35" class="reference"><a href="#cite_note-35"><span class="cite-bracket">[</span>35<span class="cite-bracket">]</span></a></sup> performs word-sense discrimination and embedding simultaneously, improving its training time, while assuming a specific number of senses for each word. In the Non-Parametric Multi-Sense Skip-Gram (NP-MSSG) this number can vary depending on each word. Combining the prior knowledge of lexical databases (e.g., <a href="WordNet" title="WordNet">WordNet</a>, <a href="Open_Mind_Common_Sense" title="Open Mind Common Sense">ConceptNet</a>, <a href="BabelNet" title="BabelNet">BabelNet</a>), word embeddings and <a href="Word_sense_disambiguation" class="mw-redirect" title="Word sense disambiguation">word sense disambiguation</a>, Most Suitable Sense Annotation (MSSA)<sup id="cite_ref-36" class="reference"><a href="#cite_note-36"><span class="cite-bracket">[</span>36<span class="cite-bracket">]</span></a></sup> labels word-senses through an unsupervised and knowledge-based approach, considering a word's context in a pre-defined sliding window. Once the words are disambiguated, they can be used in a standard word embeddings technique, so multi-sense embeddings are produced. MSSA architecture allows the disambiguation and annotation process to be performed recurrently in a self-improving manner.<sup id="cite_ref-37" class="reference"><a href="#cite_note-37"><span class="cite-bracket">[</span>37<span class="cite-bracket">]</span></a></sup>
</p><p>The use of multi-sense embeddings is known to improve performance in several NLP tasks, such as <a href="Part-of-speech_tagging" title="Part-of-speech tagging">part-of-speech tagging</a>, semantic relation identification, <a href="Semantic_relatedness" class="mw-redirect" title="Semantic relatedness">semantic relatedness</a>, <a href="Named_entity_recognition" class="mw-redirect" title="Named entity recognition">named entity recognition</a> and sentiment analysis.<sup id="cite_ref-:1_38-0" class="reference"><a href="#cite_note-:1-38"><span class="cite-bracket">[</span>38<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-39" class="reference"><a href="#cite_note-39"><span class="cite-bracket">[</span>39<span class="cite-bracket">]</span></a></sup>
</p><p>As of the late 2010s, contextually-meaningful embeddings such as <a href="ELMo" title="ELMo">ELMo</a> and <a href="BERT_(language_model)" title="BERT (language model)">BERT</a> have been developed.<sup id="cite_ref-40" class="reference"><a href="#cite_note-40"><span class="cite-bracket">[</span>40<span class="cite-bracket">]</span></a></sup> Unlike static word embeddings, these embeddings are at the token-level, in that each occurrence of a word has its own embedding. These embeddings better reflect the multi-sense nature of words, because occurrences of a word in similar contexts are situated in similar regions of BERT’s embedding space.<sup id="cite_ref-41" class="reference"><a href="#cite_note-41"><span class="cite-bracket">[</span>41<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-42" class="reference"><a href="#cite_note-42"><span class="cite-bracket">[</span>42<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading2"><h2 id="For_biological_sequences:_BioVectors">For biological sequences: BioVectors</h2></div>
<p>Word embeddings for <i>n-</i>grams in biological sequences (e.g. DNA, RNA, and Proteins) for <a href="Bioinformatics" title="Bioinformatics">bioinformatics</a> applications have been proposed by Asgari and Mofrad.<sup id="cite_ref-:0_43-0" class="reference"><a href="#cite_note-:0-43"><span class="cite-bracket">[</span>43<span class="cite-bracket">]</span></a></sup> Named bio-vectors (BioVec) to refer to biological sequences in general with protein-vectors (ProtVec) for proteins (amino-acid sequences) and gene-vectors (GeneVec) for gene sequences, this representation can be widely used in applications of deep learning in <a href="Proteomics" title="Proteomics">proteomics</a> and <a href="Genomics" title="Genomics">genomics</a>. The results presented by Asgari and Mofrad<sup id="cite_ref-:0_43-1" class="reference"><a href="#cite_note-:0-43"><span class="cite-bracket">[</span>43<span class="cite-bracket">]</span></a></sup> suggest that BioVectors can characterize biological sequences in terms of biochemical and biophysical interpretations of the underlying patterns.
</p>
<div class="mw-heading mw-heading2"><h2 id="Game_design">Game design</h2></div>
<p>Word embeddings with applications in <a href="Game_design" title="Game design">game design</a> have been proposed by Rabii and Cook<sup id="cite_ref-:2_44-0" class="reference"><a href="#cite_note-:2-44"><span class="cite-bracket">[</span>44<span class="cite-bracket">]</span></a></sup> as a way to discover <a href="Emergent_gameplay" title="Emergent gameplay">emergent gameplay</a> using logs of gameplay data. The process requires transcribing actions that occur during a game within a <a href="Formal_language" title="Formal language">formal language</a> and then using the resulting text to create word embeddings. The results presented by Rabii and Cook<sup id="cite_ref-:2_44-1" class="reference"><a href="#cite_note-:2-44"><span class="cite-bracket">[</span>44<span class="cite-bracket">]</span></a></sup> suggest that the resulting vectors can capture expert knowledge about games like <a href="Chess" title="Chess">chess</a> that are not explicitly stated in the game's rules.
</p>
<div class="mw-heading mw-heading2"><h2 id="Sentence_embeddings">Sentence embeddings</h2></div>
<style data-mw-deduplicate="TemplateStyles:r1236090951">
/* start https://en.wikipedia.org/ */
.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}
/* end https://en.wikipedia.org/ */
</style><div role="note" class="hatnote navigation-not-searchable">Main article: <a href="Sentence_embedding" title="Sentence embedding">Sentence embedding</a></div>
<p>The idea has been extended to embeddings of entire sentences or even documents, e.g. in the form of the <a href="Thought_vector" title="Thought vector">thought vectors</a> concept. In 2015, some researchers suggested "skip-thought vectors" as a means to improve the quality of <a href="Machine_translation" title="Machine translation">machine translation</a>.<sup id="cite_ref-45" class="reference"><a href="#cite_note-45"><span class="cite-bracket">[</span>45<span class="cite-bracket">]</span></a></sup> A more recent and popular approach for representing sentences is Sentence-BERT, or SentenceTransformers, which modifies pre-trained <a href="BERT_(language_model)" title="BERT (language model)">BERT</a> with the use of siamese and triplet network structures.<sup id="cite_ref-46" class="reference"><a href="#cite_note-46"><span class="cite-bracket">[</span>46<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading2"><h2 id="Software">Software</h2></div>
<p>Software for training and using word embeddings includes <a href="Tom%C3%A1%C5%A1_Mikolov" title="Tomáš Mikolov">Tomáš Mikolov</a>'s <a href="Word2vec" title="Word2vec">Word2vec</a>, Stanford University's <a href="GloVe_(machine_learning)" class="mw-redirect" title="GloVe (machine learning)">GloVe</a>,<sup id="cite_ref-47" class="reference"><a href="#cite_note-47"><span class="cite-bracket">[</span>47<span class="cite-bracket">]</span></a></sup> GN-GloVe,<sup id="cite_ref-gn-glove_48-0" class="reference"><a href="#cite_note-gn-glove-48"><span class="cite-bracket">[</span>48<span class="cite-bracket">]</span></a></sup> Flair embeddings,<sup id="cite_ref-:1_38-1" class="reference"><a href="#cite_note-:1-38"><span class="cite-bracket">[</span>38<span class="cite-bracket">]</span></a></sup> AllenNLP's <a href="ELMo" title="ELMo">ELMo</a>,<sup id="cite_ref-49" class="reference"><a href="#cite_note-49"><span class="cite-bracket">[</span>49<span class="cite-bracket">]</span></a></sup> <a href="BERT_(language_model)" title="BERT (language model)">BERT</a>,<sup id="cite_ref-50" class="reference"><a href="#cite_note-50"><span class="cite-bracket">[</span>50<span class="cite-bracket">]</span></a></sup> <a href="FastText" title="FastText">fastText</a>, <a href="Gensim" title="Gensim">Gensim</a>,<sup id="cite_ref-51" class="reference"><a href="#cite_note-51"><span class="cite-bracket">[</span>51<span class="cite-bracket">]</span></a></sup> Indra,<sup id="cite_ref-52" class="reference"><a href="#cite_note-52"><span class="cite-bracket">[</span>52<span class="cite-bracket">]</span></a></sup> and <a href="Deeplearning4j" title="Deeplearning4j">Deeplearning4j</a>. <a href="Principal_Component_Analysis" class="mw-redirect" title="Principal Component Analysis">Principal Component Analysis</a> (PCA) and <a href="T-Distributed_Stochastic_Neighbour_Embedding" class="mw-redirect" title="T-Distributed Stochastic Neighbour Embedding">T-Distributed Stochastic Neighbour Embedding</a> (t-SNE) are both used to reduce the dimensionality of word vector spaces and visualize word embeddings and <a href="Word-sense_induction" title="Word-sense induction">clusters</a>.<sup id="cite_ref-53" class="reference"><a href="#cite_note-53"><span class="cite-bracket">[</span>53<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading3"><h3 id="Examples_of_application">Examples of application</h3></div>
<p>For instance, the fastText is also used to calculate word embeddings for <a href="Text_corpora" class="mw-redirect" title="Text corpora">text corpora</a> in <a href="Sketch_Engine" title="Sketch Engine">Sketch Engine</a> that are available online.<sup id="cite_ref-54" class="reference"><a href="#cite_note-54"><span class="cite-bracket">[</span>54<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading2"><h2 id="Ethical_implications">Ethical implications</h2></div>
<p>Word embeddings may contain the biases and stereotypes contained in the trained dataset, as Bolukbasi et al. points out in the 2016 paper “Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings” that a publicly available (and popular) word2vec embedding trained on Google News texts (a commonly used data corpus), which consists of text written by professional journalists, still shows disproportionate word associations reflecting gender and racial biases when extracting word analogies.<sup id="cite_ref-55" class="reference"><a href="#cite_note-55"><span class="cite-bracket">[</span>55<span class="cite-bracket">]</span></a></sup> For example, one of the analogies generated using the aforementioned word embedding is “man is to computer programmer as woman is to homemaker”.<sup id="cite_ref-56" class="reference"><a href="#cite_note-56"><span class="cite-bracket">[</span>56<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-57" class="reference"><a href="#cite_note-57"><span class="cite-bracket">[</span>57<span class="cite-bracket">]</span></a></sup>
</p><p>Research done by Jieyu Zhou et al. shows that the applications of these trained word embeddings without careful oversight likely perpetuates existing bias in society, which is introduced through unaltered training data. Furthermore, word embeddings can even amplify these biases .<sup id="cite_ref-58" class="reference"><a href="#cite_note-58"><span class="cite-bracket">[</span>58<span class="cite-bracket">]</span></a></sup><sup id="cite_ref-59" class="reference"><a href="#cite_note-59"><span class="cite-bracket">[</span>59<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading2"><h2 id="See_also">See also</h2></div>
<ul><li><a href="Embedding_(machine_learning)" title="Embedding (machine learning)">Embedding (machine learning)</a></li>
<li><a href="Brown_clustering" title="Brown clustering">Brown clustering</a></li>
<li><a href="Distributional%E2%80%93relational_database" title="Distributional–relational database">Distributional–relational database</a></li></ul>
<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>
<style data-mw-deduplicate="TemplateStyles:r1239543626">
/* start https://en.wikipedia.org/ */
.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}
/* end https://en.wikipedia.org/ */
</style><div class="reflist">
<div class="mw-references-wrap mw-references-columns"><ol class="references">
<li id="cite_note-1"><span class="mw-cite-backlink"><b><a href="#cite_ref-1">^</a></b></span> <span class="reference-text"><style data-mw-deduplicate="TemplateStyles:r1238218222">
/* start https://en.wikipedia.org/ */
.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}
/* end https://en.wikipedia.org/ */
</style><cite id="CITEREFJurafskyH._James2000" class="citation book cs1">Jurafsky, Daniel; H. James, Martin (2000). <a rel="nofollow" class="external text" href="https://web.stanford.edu/~jurafsky/slp3/"><i>Speech and language processing : an introduction to natural language processing, computational linguistics, and speech recognition</i></a>. Upper Saddle River, N.J.: Prentice Hall. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-13-095069-7</bdi>.</cite></span>
</li>
<li id="cite_note-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-2">^</a></b></span> <span class="reference-text"><cite id="CITEREFMikolovSutskeverChenCorrado2013" class="citation arxiv cs1">Mikolov, Tomas; Sutskever, Ilya; Chen, Kai; Corrado, Greg; Dean, Jeffrey (2013). "Distributed Representations of Words and Phrases and their Compositionality". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1310.4546">1310.4546</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-3"><span class="mw-cite-backlink"><b><a href="#cite_ref-3">^</a></b></span> <span class="reference-text"><cite id="CITEREFLebretCollobert2013" class="citation book cs1">Lebret, Rémi; Collobert, Ronan (2013). "Word Emdeddings through Hellinger PCA". <i>Conference of the European Chapter of the Association for Computational Linguistics (EACL)</i>. Vol. 2014. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1312.5542">1312.5542</a></span>.</cite></span>
</li>
<li id="cite_note-4"><span class="mw-cite-backlink"><b><a href="#cite_ref-4">^</a></b></span> <span class="reference-text"><cite id="CITEREFLevyGoldberg2014" class="citation conference cs1">Levy, Omer; Goldberg, Yoav (2014). <a rel="nofollow" class="external text" href="http://papers.nips.cc/paper/5477-neural-word-embedding-as-implicit-matrix-factorization.pdf"><i>Neural Word Embedding as Implicit Matrix Factorization</i></a> <span class="cs1-format">(PDF)</span>. NIPS.</cite></span>
</li>
<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text"><cite id="CITEREFLiXu2015" class="citation conference cs1">Li, Yitan; Xu, Linli (2015). <a rel="nofollow" class="external text" href="http://ijcai.org/papers15/Papers/IJCAI15-513.pdf"><i>Word Embedding Revisited: A New Representation Learning and Explicit Matrix Factorization Perspective</i></a> <span class="cs1-format">(PDF)</span>. Int'l J. Conf. on Artificial Intelligence (IJCAI).</cite></span>
</li>
<li id="cite_note-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-6">^</a></b></span> <span class="reference-text"><cite id="CITEREFGloberson2007" class="citation journal cs1">Globerson, Amir (2007). <a rel="nofollow" class="external text" href="http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/34951.pdf">"Euclidean Embedding of Co-occurrence Data"</a> <span class="cs1-format">(PDF)</span>. <i>Journal of Machine Learning Research</i>.</cite></span>
</li>
<li id="cite_note-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-7">^</a></b></span> <span class="reference-text"><cite id="CITEREFQureshiGreene2018" class="citation journal cs1">Qureshi, M. Atif; Greene, Derek (2018-06-04). "EVE: explainable vector based embedding technique using Wikipedia". <i>Journal of Intelligent Information Systems</i>. <b>53</b>: <span class="nowrap">137–</span>165. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1702.06891">1702.06891</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs10844-018-0511-x">10.1007/s10844-018-0511-x</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0925-9902">0925-9902</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:10656055">10656055</a>.</cite></span>
</li>
<li id="cite_note-8"><span class="mw-cite-backlink"><b><a href="#cite_ref-8">^</a></b></span> <span class="reference-text"><cite id="CITEREFLevyGoldberg2014" class="citation conference cs1">Levy, Omer; Goldberg, Yoav (2014). <a rel="nofollow" class="external text" href="https://levyomer.files.wordpress.com/2014/04/linguistic-regularities-in-sparse-and-explicit-word-representations-conll-2014.pdf"><i>Linguistic Regularities in Sparse and Explicit Word Representations</i></a> <span class="cs1-format">(PDF)</span>. CoNLL. pp. <span class="nowrap">171–</span>180.</cite></span>
</li>
<li id="cite_note-9"><span class="mw-cite-backlink"><b><a href="#cite_ref-9">^</a></b></span> <span class="reference-text"><cite id="CITEREFSocherBauerManningNg2013" class="citation conference cs1">Socher, Richard; Bauer, John; Manning, Christopher; Ng, Andrew (2013). <a rel="nofollow" class="external text" href="https://web.archive.org/web/20160811041232/http://www.socher.org/uploads/Main/SocherBauerManningNg_ACL2013.pdf"><i>Parsing with compositional vector grammars</i></a> <span class="cs1-format">(PDF)</span>. Proc. ACL Conf. Archived from <a rel="nofollow" class="external text" href="http://www.socher.org/uploads/Main/SocherBauerManningNg_ACL2013.pdf">the original</a> <span class="cs1-format">(PDF)</span> on 2016-08-11<span class="reference-accessdate">. Retrieved <span class="nowrap">2014-08-14</span></span>.</cite></span>
</li>
<li id="cite_note-10"><span class="mw-cite-backlink"><b><a href="#cite_ref-10">^</a></b></span> <span class="reference-text"><cite id="CITEREFSocherPerelyginWuChuang2013" class="citation conference cs1">Socher, Richard; Perelygin, Alex; Wu, Jean; Chuang, Jason; Manning, Chris; Ng, Andrew; Potts, Chris (2013). <a rel="nofollow" class="external text" href="http://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf"><i>Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank</i></a> <span class="cs1-format">(PDF)</span>. EMNLP.</cite></span>
</li>
<li id="cite_note-11"><span class="mw-cite-backlink"><b><a href="#cite_ref-11">^</a></b></span> <span class="reference-text"><cite id="CITEREFSahlgren" class="citation web cs1">Sahlgren, Magnus. <a rel="nofollow" class="external text" href="https://www.linkedin.com/pulse/brief-history-word-embeddings-some-clarifications-magnus-sahlgren/">"A brief history of word embeddings"</a>.</cite></span>
</li>
<li id="cite_note-12"><span class="mw-cite-backlink"><b><a href="#cite_ref-12">^</a></b></span> <span class="reference-text"><cite id="CITEREFFirth1957" class="citation journal cs1">Firth, J.R. (1957). "A synopsis of linguistic theory 1930–1955". <i>Studies in Linguistic Analysis</i>: <span class="nowrap">1–</span>32.</cite> Reprinted in <cite id="CITEREFF.R._Palmer1968" class="citation book cs1">F.R. Palmer, ed. (1968). <i>Selected Papers of J.R. Firth 1952–1959</i>. London: Longman.</cite><span class="cs1-maint citation-comment"><code class="cs1-code">{{cite book}}</code>: CS1 maint: publisher location (link)</span></span>
</li>
<li id="cite_note-13"><span class="mw-cite-backlink"><b><a href="#cite_ref-13">^</a></b></span> <span class="reference-text"><cite id="CITEREFLuhn1953" class="citation journal cs1">Luhn, H.P. (1953). "A New Method of Recording and Searching Information". <i>American Documentation</i>. <b>4</b>: <span class="nowrap">14–</span>16. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2Fasi.5090040104">10.1002/asi.5090040104</a>.</cite></span>
</li>
<li id="cite_note-14"><span class="mw-cite-backlink"><b><a href="#cite_ref-14">^</a></b></span> <span class="reference-text"><cite id="CITEREFOsgoodSuciTannenbaum1957" class="citation book cs1">Osgood, C.E.; Suci, G.J.; Tannenbaum, P.H. (1957). <i>The Measurement of Meaning</i>. University of Illinois Press.</cite></span>
</li>
<li id="cite_note-Salton_original-15"><span class="mw-cite-backlink"><b><a href="#cite_ref-Salton_original_15-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFSalton1962" class="citation book cs1">Salton, Gerard (1962). "Some experiments in the generation of word and document associations". <i>Proceedings of the December 4-6, 1962, fall joint computer conference on - AFIPS '62 (Fall)</i>. pp. <span class="nowrap">234–</span>250. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F1461518.1461544">10.1145/1461518.1461544</a></span>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781450378796</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:9937095">9937095</a>.</cite> <span class="cs1-hidden-error citation-comment"><code class="cs1-code">{{cite book}}</code>: </span><span class="cs1-hidden-error citation-comment">ISBN / Date incompatibility (help)</span></span>
</li>
<li id="cite_note-SaltonEA_CACM-16"><span class="mw-cite-backlink"><b><a href="#cite_ref-SaltonEA_CACM_16-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFSaltonWongYang1975" class="citation journal cs1">Salton, Gerard; Wong, A; Yang, C S (1975). "A Vector Space Model for Automatic Indexing". <i>Communications of the ACM</i>. <b>18</b> (11): <span class="nowrap">613–</span>620. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F361219.361220">10.1145/361219.361220</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://hdl.handle.net/1813%2F6057">1813/6057</a></span>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:6473756">6473756</a>.</cite></span>
</li>
<li id="cite_note-17"><span class="mw-cite-backlink"><b><a href="#cite_ref-17">^</a></b></span> <span class="reference-text"><cite id="CITEREFDubin2004" class="citation web cs1">Dubin, David (2004). <a rel="nofollow" class="external text" href="https://web.archive.org/web/20201018193507/https://www.thefreelibrary.com/The+most+influential+paper+Gerard+Salton+never+wrote.-a0125151308">"The most influential paper Gerard Salton never wrote"</a>. Archived from <a rel="nofollow" class="external text" href="https://www.thefreelibrary.com/The+most+influential+paper+Gerard+Salton+never+wrote.-a0125151308">the original</a> on 18 October 2020<span class="reference-accessdate">. Retrieved <span class="nowrap">18 October</span> 2020</span>.</cite></span>
</li>
<li id="cite_note-18"><span class="mw-cite-backlink"><b><a href="#cite_ref-18">^</a></b></span> <span class="reference-text">Kanerva, Pentti, Kristoferson, Jan and Holst, Anders (2000): <a rel="nofollow" class="external text" href="https://cloudfront.escholarship.org/dist/prd/content/qt5644k0w6/qt5644k0w6.pdf">Random Indexing of Text Samples for Latent Semantic Analysis</a>, Proceedings of the 22nd Annual Conference of the Cognitive Science Society, p. 1036. Mahwah, New Jersey: Erlbaum, 2000.</span>
</li>
<li id="cite_note-19"><span class="mw-cite-backlink"><b><a href="#cite_ref-19">^</a></b></span> <span class="reference-text"><cite id="CITEREFKarlgrenSahlgren2001" class="citation journal cs1">Karlgren, Jussi; Sahlgren, Magnus (2001). Uesaka, Yoshinori; Kanerva, Pentti; Asoh, Hideki (eds.). "From words to understanding". <i>Foundations of Real-World Intelligence</i>. CSLI Publications: <span class="nowrap">294–</span>308.</cite></span>
</li>
<li id="cite_note-20"><span class="mw-cite-backlink"><b><a href="#cite_ref-20">^</a></b></span> <span class="reference-text">Sahlgren, Magnus (2005) <a rel="nofollow" class="external text" href="http://eprints.sics.se/221/1/RI_intro.pdf">An Introduction to Random Indexing</a>, Proceedings of the Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology and Knowledge Engineering, TKE 2005, August 16, Copenhagen, Denmark</span>
</li>
<li id="cite_note-21"><span class="mw-cite-backlink"><b><a href="#cite_ref-21">^</a></b></span> <span class="reference-text">Sahlgren, Magnus, Holst, Anders and Pentti Kanerva (2008) <a rel="nofollow" class="external text" href="http://eprints.sics.se/3436/01/permutationsCogSci08.pdf">Permutations as a Means to Encode Order in Word Space</a>, In Proceedings of the 30th Annual Conference of the Cognitive Science Society: 1300–1305.</span>
</li>
<li id="cite_note-22"><span class="mw-cite-backlink"><b><a href="#cite_ref-22">^</a></b></span> <span class="reference-text"><cite id="CITEREFBengioRéjeanPascal2000" class="citation journal cs1">Bengio, Yoshua; Réjean, Ducharme; Pascal, Vincent (2000). <a rel="nofollow" class="external text" href="https://proceedings.neurips.cc/paper_files/paper/2000/file/728f206c2a01bf572b5940d7d9a8fa4c-Paper.pdf">"A Neural Probabilistic Language Model"</a> <span class="cs1-format">(PDF)</span>. <i>NeurIPS</i>.</cite></span>
</li>
<li id="cite_note-23"><span class="mw-cite-backlink"><b><a href="#cite_ref-23">^</a></b></span> <span class="reference-text"><cite id="CITEREFBengioDucharmeVincentJauvin2003" class="citation journal cs1"><a href="Yoshua_Bengio" title="Yoshua Bengio">Bengio, Yoshua</a>; Ducharme, Réjean; Vincent, Pascal; Jauvin, Christian (2003). <a rel="nofollow" class="external text" href="https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf">"A Neural Probabilistic Language Model"</a> <span class="cs1-format">(PDF)</span>. <i>Journal of Machine Learning Research</i>. <b>3</b>: <span class="nowrap">1137–</span>1155.</cite></span>
</li>
<li id="cite_note-24"><span class="mw-cite-backlink"><b><a href="#cite_ref-24">^</a></b></span> <span class="reference-text"><cite id="CITEREFBengioSchwenkSenécalMorin2006" class="citation book cs1">Bengio, Yoshua; Schwenk, Holger; Senécal, Jean-Sébastien; Morin, Fréderic; Gauvain, Jean-Luc (2006). "A Neural Probabilistic Language Model". <i>Studies in Fuzziness and Soft Computing</i>. Vol. 194. Springer. pp. <span class="nowrap">137–</span>186. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F3-540-33486-6_6">10.1007/3-540-33486-6_6</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-3-540-30609-2</bdi>.</cite></span>
</li>
<li id="cite_note-25"><span class="mw-cite-backlink"><b><a href="#cite_ref-25">^</a></b></span> <span class="reference-text"><cite id="CITEREFVinkourovCristianiniShawe-Taylor2002" class="citation conference cs1">Vinkourov, Alexei; Cristianini, Nello; Shawe-Taylor, John (2002). <a rel="nofollow" class="external text" href="https://proceedings.neurips.cc/paper/2002/file/d5e2fbef30a4eb668a203060ec8e5eef-Paper.pdf"><i>Inferring a semantic representation of text via cross-language correlation analysis</i></a> <span class="cs1-format">(PDF)</span>. Advances in Neural Information Processing Systems. Vol. 15.</cite></span>
</li>
<li id="cite_note-26"><span class="mw-cite-backlink"><b><a href="#cite_ref-26">^</a></b></span> <span class="reference-text"><cite id="CITEREFLavelliSebastianiZanoli2004" class="citation conference cs1">Lavelli, Alberto; Sebastiani, Fabrizio; Zanoli, Roberto (2004). <i>Distributional term representations: an experimental comparison</i>. 13th ACM International Conference on Information and Knowledge Management. pp. <span class="nowrap">615–</span>624. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F1031171.1031284">10.1145/1031171.1031284</a>.</cite></span>
</li>
<li id="cite_note-27"><span class="mw-cite-backlink"><b><a href="#cite_ref-27">^</a></b></span> <span class="reference-text"><cite id="CITEREFRoweisSaul2000" class="citation journal cs1">Roweis, Sam T.; Saul, Lawrence K. (2000). "Nonlinear Dimensionality Reduction by Locally Linear Embedding". <i>Science</i>. <b>290</b> (5500): <span class="nowrap">2323–</span>6. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2000Sci...290.2323R">2000Sci...290.2323R</a>. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.111.3313">10.1.1.111.3313</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1126%2Fscience.290.5500.2323">10.1126/science.290.5500.2323</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/11125150">11125150</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:5987139">5987139</a>.</cite></span>
</li>
<li id="cite_note-28"><span class="mw-cite-backlink"><b><a href="#cite_ref-28">^</a></b></span> <span class="reference-text"><a href="https://he.wikipedia.org/wiki/%D7%99%D7%94%D7%95%D7%A9%D7%A2_%D7%91%D7%A0%D7%92%27%D7%99%D7%95" class="extiw external" title="he:יהושע בנג'יו">he:יהושע בנג'יו</a></span>
</li>
<li id="cite_note-29"><span class="mw-cite-backlink"><b><a href="#cite_ref-29">^</a></b></span> <span class="reference-text"><cite id="CITEREFMorinBengio2005" class="citation book cs1">Morin, Fredric; Bengio, Yoshua (2005). <a rel="nofollow" class="external text" href="http://proceedings.mlr.press/r5/morin05a/morin05a.pdf">"Hierarchical probabilistic neural network language model"</a> <span class="cs1-format">(PDF)</span>. In Cowell, Robert G.; Ghahramani, Zoubin (eds.). <i>Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics</i>. Proceedings of Machine Learning Research. Vol. R5. pp. <span class="nowrap">246–</span>252.</cite></span>
</li>
<li id="cite_note-30"><span class="mw-cite-backlink"><b><a href="#cite_ref-30">^</a></b></span> <span class="reference-text"><cite id="CITEREFMnihHinton2009" class="citation journal cs1 cs1-prop-long-vol">Mnih, Andriy; Hinton, Geoffrey (2009). <a rel="nofollow" class="external text" href="http://papers.nips.cc/paper/3583-a-scalable-hierarchical-distributed-language-model">"A Scalable Hierarchical Distributed Language Model"</a>. <i>Advances in Neural Information Processing Systems</i>. 21 (NIPS 2008). Curran Associates, Inc.: <span class="nowrap">1081–</span>1088.</cite></span>
</li>
<li id="cite_note-31"><span class="mw-cite-backlink"><b><a href="#cite_ref-31">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://code.google.com/archive/p/word2vec/">"word2vec"</a>. <i>Google Code Archive</i><span class="reference-accessdate">. Retrieved <span class="nowrap">23 July</span> 2021</span>.</cite></span>
</li>
<li id="cite_note-32"><span class="mw-cite-backlink"><b><a href="#cite_ref-32">^</a></b></span> <span class="reference-text"><cite id="CITEREFReisingerMooney2010" class="citation book cs1 cs1-prop-long-vol">Reisinger, Joseph; Mooney, Raymond J. (2010). <a rel="nofollow" class="external text" href="https://www.aclweb.org/anthology/N10-1013/"><i>Multi-Prototype Vector-Space Models of Word Meaning</i></a>. Vol. Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Los Angeles, California: Association for Computational Linguistics. pp. <span class="nowrap">109–</span>117. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-932432-65-7</bdi><span class="reference-accessdate">. Retrieved <span class="nowrap">October 25,</span> 2019</span>.</cite></span>
</li>
<li id="cite_note-33"><span class="mw-cite-backlink"><b><a href="#cite_ref-33">^</a></b></span> <span class="reference-text"><cite id="CITEREFHuang,_Eric.2012" class="citation book cs1">Huang, Eric. (2012). <i>Improving word representations via global context and multiple word prototypes</i>. <a href="OCLC_(identifier)" class="mw-redirect" title="OCLC (identifier)">OCLC</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/oclc/857900050">857900050</a>.</cite></span>
</li>
<li id="cite_note-34"><span class="mw-cite-backlink"><b><a href="#cite_ref-34">^</a></b></span> <span class="reference-text"><cite id="CITEREFCamacho-ColladosPilehvar2018" class="citation arxiv cs1">Camacho-Collados, Jose; Pilehvar, Mohammad Taher (2018). "From Word to Sense Embeddings: A Survey on Vector Representations of Meaning". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1805.04032">1805.04032</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-35"><span class="mw-cite-backlink"><b><a href="#cite_ref-35">^</a></b></span> <span class="reference-text"><cite id="CITEREFNeelakantanShankarPassosMcCallum2014" class="citation book cs1">Neelakantan, Arvind; Shankar, Jeevan; Passos, Alexandre; McCallum, Andrew (2014). "Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space". <i>Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)</i>. Stroudsburg, PA, USA: Association for Computational Linguistics. pp. <span class="nowrap">1059–</span>1069. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1504.06654">1504.06654</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.3115%2Fv1%2Fd14-1113">10.3115/v1/d14-1113</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:15251438">15251438</a>.</cite></span>
</li>
<li id="cite_note-36"><span class="mw-cite-backlink"><b><a href="#cite_ref-36">^</a></b></span> <span class="reference-text"><cite id="CITEREFRuasGroskyAizawa2019" class="citation journal cs1">Ruas, Terry; Grosky, William; Aizawa, Akiko (2019-12-01). "Multi-sense embeddings through a word sense disambiguation process". <i>Expert Systems with Applications</i>. <b>136</b>: <span class="nowrap">288–</span>303. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/2101.08700">2101.08700</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.eswa.2019.06.026">10.1016/j.eswa.2019.06.026</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://hdl.handle.net/2027.42%2F145475">2027.42/145475</a></span>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/0957-4174">0957-4174</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:52225306">52225306</a>.</cite></span>
</li>
<li id="cite_note-37"><span class="mw-cite-backlink"><b><a href="#cite_ref-37">^</a></b></span> <span class="reference-text"><cite id="CITEREFAgrePetrovKeskinova2019" class="citation journal cs1">Agre, Gennady; Petrov, Daniel; Keskinova, Simona (2019-03-01). <a rel="nofollow" class="external text" href="https://doi.org/10.3390%2Finfo10030097">"Word Sense Disambiguation Studio: A Flexible System for WSD Feature Extraction"</a>. <i>Information</i>. <b>10</b> (3): 97. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.3390%2Finfo10030097">10.3390/info10030097</a></span>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/2078-2489">2078-2489</a>.</cite></span>
</li>
<li id="cite_note-:1-38"><span class="mw-cite-backlink">^ <a href="#cite_ref-:1_38-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-:1_38-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFAkbikBlytheVollgraf2018" class="citation journal cs1">Akbik, Alan; Blythe, Duncan; Vollgraf, Roland (2018). <a rel="nofollow" class="external text" href="https://www.aclweb.org/anthology/C18-1139">"Contextual String Embeddings for Sequence Labeling"</a>. <i>Proceedings of the 27th International Conference on Computational Linguistics</i>. Santa Fe, New Mexico, USA: Association for Computational Linguistics: <span class="nowrap">1638–</span>1649.</cite></span>
</li>
<li id="cite_note-39"><span class="mw-cite-backlink"><b><a href="#cite_ref-39">^</a></b></span> <span class="reference-text"><cite id="CITEREFLiJurafsky2015" class="citation book cs1">Li, Jiwei; Jurafsky, Dan (2015). "Do Multi-Sense Embeddings Improve Natural Language Understanding?". <i>Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing</i>. Stroudsburg, PA, USA: Association for Computational Linguistics. pp. <span class="nowrap">1722–</span>1732. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1506.01070">1506.01070</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.18653%2Fv1%2Fd15-1200">10.18653/v1/d15-1200</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:6222768">6222768</a>.</cite></span>
</li>
<li id="cite_note-40"><span class="mw-cite-backlink"><b><a href="#cite_ref-40">^</a></b></span> <span class="reference-text"><cite id="CITEREFDevlinChangLeeToutanova2019" class="citation journal cs1">Devlin, Jacob; Chang, Ming-Wei; Lee, Kenton; Toutanova, Kristina (June 2019). <span class="id-lock-subscription" title="Paid subscription required"><a rel="nofollow" class="external text" href="https://aclanthology.org/N19-1423/">"Proceedings of the 2019 Conference of the North"</a></span>. <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)</i>. Association for Computational Linguistics: <span class="nowrap">4171–</span>4186. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.18653%2Fv1%2FN19-1423">10.18653/v1/N19-1423</a></span>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:52967399">52967399</a>.</cite></span>
</li>
<li id="cite_note-41"><span class="mw-cite-backlink"><b><a href="#cite_ref-41">^</a></b></span> <span class="reference-text">Lucy, Li, and David Bamman. "Characterizing English variation across social media communities with BERT." Transactions of the Association for Computational Linguistics 9 (2021): 538-556.</span>
</li>
<li id="cite_note-42"><span class="mw-cite-backlink"><b><a href="#cite_ref-42">^</a></b></span> <span class="reference-text">Reif, Emily, Ann Yuan, Martin Wattenberg, Fernanda B. Viegas, Andy Coenen, Adam Pearce, and Been Kim. "Visualizing and measuring the geometry of BERT." Advances in Neural Information Processing Systems 32 (2019).</span>
</li>
<li id="cite_note-:0-43"><span class="mw-cite-backlink">^ <a href="#cite_ref-:0_43-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-:0_43-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFAsgariMofrad2015" class="citation journal cs1">Asgari, Ehsaneddin; Mofrad, Mohammad R.K. (2015). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4640716">"Continuous Distributed Representation of Biological Sequences for Deep Proteomics and Genomics"</a>. <i>PLOS ONE</i>. <b>10</b> (11): e0141287. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1503.05140">1503.05140</a></span>. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2015PLoSO..1041287A">2015PLoSO..1041287A</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1371%2Fjournal.pone.0141287">10.1371/journal.pone.0141287</a></span>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4640716">4640716</a></span>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/26555596">26555596</a>.</cite></span>
</li>
<li id="cite_note-:2-44"><span class="mw-cite-backlink">^ <a href="#cite_ref-:2_44-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-:2_44-1"><sup><i><b>b</b></i></sup></a></span> <span class="reference-text"><cite id="CITEREFRabiiCook2021" class="citation journal cs1">Rabii, Younès; Cook, Michael (2021-10-04). <a rel="nofollow" class="external text" href="https://ojs.aaai.org/index.php/AIIDE/article/view/18907">"Revealing Game Dynamics via Word Embeddings of Gameplay Data"</a>. <i>Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment</i>. <b>17</b> (1): <span class="nowrap">187–</span>194. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1609%2Faiide.v17i1.18907">10.1609/aiide.v17i1.18907</a></span>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/2334-0924">2334-0924</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:248175634">248175634</a>.</cite></span>
</li>
<li id="cite_note-45"><span class="mw-cite-backlink"><b><a href="#cite_ref-45">^</a></b></span> <span class="reference-text"><cite id="CITEREFKirosZhuSalakhutdinovZemel2015" class="citation arxiv cs1">Kiros, Ryan; Zhu, Yukun; Salakhutdinov, Ruslan; Zemel, Richard S.; Torralba, Antonio; Urtasun, Raquel; Fidler, Sanja (2015). "skip-thought vectors". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1506.06726">1506.06726</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-46"><span class="mw-cite-backlink"><b><a href="#cite_ref-46">^</a></b></span> <span class="reference-text">Reimers, Nils, and Iryna Gurevych. "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks." In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982-3992. 2019.</span>
</li>
<li id="cite_note-47"><span class="mw-cite-backlink"><b><a href="#cite_ref-47">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://nlp.stanford.edu/projects/glove/">"GloVe"</a>.</cite></span>
</li>
<li id="cite_note-gn-glove-48"><span class="mw-cite-backlink"><b><a href="#cite_ref-gn-glove_48-0">^</a></b></span> <span class="reference-text"><cite id="CITEREFZhao2018" class="citation arxiv cs1">Zhao, Jieyu; et al. (2018) (2018). "Learning Gender-Neutral Word Embeddings". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1809.01496">1809.01496</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-49"><span class="mw-cite-backlink"><b><a href="#cite_ref-49">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://allennlp.org/elmo">"Elmo"</a>. 16 October 2024.</cite></span>
</li>
<li id="cite_note-50"><span class="mw-cite-backlink"><b><a href="#cite_ref-50">^</a></b></span> <span class="reference-text"><cite id="CITEREFPiresSchlingerGarrette2019" class="citation arxiv cs1">Pires, Telmo; Schlinger, Eva; Garrette, Dan (2019-06-04). "How multilingual is Multilingual BERT?". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1906.01502">1906.01502</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-51"><span class="mw-cite-backlink"><b><a href="#cite_ref-51">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="http://radimrehurek.com/gensim/">"Gensim"</a>.</cite></span>
</li>
<li id="cite_note-52"><span class="mw-cite-backlink"><b><a href="#cite_ref-52">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://github.com/Lambda-3/Indra">"Indra"</a>. <i><a href="GitHub" title="GitHub">GitHub</a></i>. 2018-10-25.</cite></span>
</li>
<li id="cite_note-53"><span class="mw-cite-backlink"><b><a href="#cite_ref-53">^</a></b></span> <span class="reference-text"><cite id="CITEREFGhassemiMarkNemati2015" class="citation book cs1">Ghassemi, Mohammad; Mark, Roger; Nemati, Shamim (2015). <a rel="nofollow" class="external text" href="http://www.cinc.org/archives/2015/pdf/0629.pdf">"A visualization of evolving clinical sentiment using vector representations of clinical notes"</a> <span class="cs1-format">(PDF)</span>. <i>2015 Computing in Cardiology Conference (CinC)</i>. Vol. 2015. pp. <span class="nowrap">629–</span>632. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FCIC.2015.7410989">10.1109/CIC.2015.7410989</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-5090-0685-4</bdi>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5070922">5070922</a></span>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/27774487">27774487</a>.</cite></span>
</li>
<li id="cite_note-54"><span class="mw-cite-backlink"><b><a href="#cite_ref-54">^</a></b></span> <span class="reference-text"><cite class="citation web cs1"><a rel="nofollow" class="external text" href="https://web.archive.org/web/20180208004241/https://embeddings.sketchengine.co.uk/">"Embedding Viewer"</a>. <i>Embedding Viewer</i>. Lexical Computing. Archived from <a rel="nofollow" class="external text" href="https://embeddings.sketchengine.co.uk/">the original</a> on 8 February 2018<span class="reference-accessdate">. Retrieved <span class="nowrap">7 Feb</span> 2018</span>.</cite></span>
</li>
<li id="cite_note-55"><span class="mw-cite-backlink"><b><a href="#cite_ref-55">^</a></b></span> <span class="reference-text"><cite id="CITEREFBolukbasiChangZouSaligrama2016" class="citation arxiv cs1">Bolukbasi, Tolga; Chang, Kai-Wei; Zou, James; Saligrama, Venkatesh; Kalai, Adam (2016). "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1607.06520">1607.06520</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-56"><span class="mw-cite-backlink"><b><a href="#cite_ref-56">^</a></b></span> <span class="reference-text"><cite id="CITEREFBolukbasiChangZouSaligrama2016" class="citation arxiv cs1">Bolukbasi, Tolga; Chang, Kai-Wei; Zou, James; Saligrama, Venkatesh; Kalai, Adam (2016-07-21). "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1607.06520">1607.06520</a></span> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/cs.CL">cs.CL</a>].</cite></span>
</li>
<li id="cite_note-57"><span class="mw-cite-backlink"><b><a href="#cite_ref-57">^</a></b></span> <span class="reference-text"><cite id="CITEREFDiengRuizBlei2020" class="citation journal cs1">Dieng, Adji B.; Ruiz, Francisco J. R.; Blei, David M. (2020). <a rel="nofollow" class="external text" href="https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00325/96463/Topic-Modeling-in-Embedding-Spaces">"Topic Modeling in Embedding Spaces"</a>. <i>Transactions of the Association for Computational Linguistics</i>. <b>8</b>: <span class="nowrap">439–</span>453. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://arxiv.org/abs/1907.04907">1907.04907</a></span>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1162%2Ftacl_a_00325">10.1162/tacl_a_00325</a>.</cite></span>
</li>
<li id="cite_note-58"><span class="mw-cite-backlink"><b><a href="#cite_ref-58">^</a></b></span> <span class="reference-text"><cite id="CITEREFZhaoWangYatskarOrdonez2017" class="citation book cs1">Zhao, Jieyu; Wang, Tianlu; Yatskar, Mark; Ordonez, Vicente; Chang, Kai-Wei (2017). <a rel="nofollow" class="external text" href="https://aclanthology.org/D17-1323/">"Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints"</a>. <i>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</i>. pp. <span class="nowrap">2979–</span>2989. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.18653%2Fv1%2FD17-1323">10.18653/v1/D17-1323</a>.</cite></span>
</li>
<li id="cite_note-59"><span class="mw-cite-backlink"><b><a href="#cite_ref-59">^</a></b></span> <span class="reference-text"><cite id="CITEREFPetreskiHashim2022" class="citation journal cs1">Petreski, Davor; Hashim, Ibrahim C. (2022-05-26). <a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs00146-022-01443-w">"Word embeddings are biased. But whose bias are they reflecting?"</a>. <i>AI & Society</i>. <b>38</b> (2): <span class="nowrap">975–</span>982. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<span class="id-lock-free" title="Freely accessible"><a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs00146-022-01443-w">10.1007/s00146-022-01443-w</a></span>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1435-5655">1435-5655</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:249112516">249112516</a>.</cite></span>
</li>
</ol></div></div>
<div class="navbox-styles"><style data-mw-deduplicate="TemplateStyles:r1236075235">
/* start https://en.wikipedia.org/ */
.mw-parser-output .navbox{box-sizing:border-box;border:1px solid #a2a9b1;width:100%;clear:both;font-size:88%;text-align:center;padding:1px;margin:1em auto 0}.mw-parser-output .navbox .navbox{margin-top:0}.mw-parser-output .navbox+.navbox,.mw-parser-output .navbox+.navbox-styles+.navbox{margin-top:-1px}.mw-parser-output .navbox-inner,.mw-parser-output .navbox-subgroup{width:100%}.mw-parser-output .navbox-group,.mw-parser-output .navbox-title,.mw-parser-output .navbox-abovebelow{padding:0.25em 1em;line-height:1.5em;text-align:center}.mw-parser-output .navbox-group{white-space:nowrap;text-align:right}.mw-parser-output .navbox,.mw-parser-output .navbox-subgroup{background-color:#fdfdfd}.mw-parser-output .navbox-list{line-height:1.5em;border-color:#fdfdfd}.mw-parser-output .navbox-list-with-group{text-align:left;border-left-width:2px;border-left-style:solid}.mw-parser-output tr+tr>.navbox-abovebelow,.mw-parser-output tr+tr>.navbox-group,.mw-parser-output tr+tr>.navbox-image,.mw-parser-output tr+tr>.navbox-list{border-top:2px solid #fdfdfd}.mw-parser-output .navbox-title{background-color:#ccf}.mw-parser-output .navbox-abovebelow,.mw-parser-output .navbox-group,.mw-parser-output .navbox-subgroup .navbox-title{background-color:#ddf}.mw-parser-output .navbox-subgroup .navbox-group,.mw-parser-output .navbox-subgroup .navbox-abovebelow{background-color:#e6e6ff}.mw-parser-output .navbox-even{background-color:#f7f7f7}.mw-parser-output .navbox-odd{background-color:transparent}.mw-parser-output .navbox .hlist td dl,.mw-parser-output .navbox .hlist td ol,.mw-parser-output .navbox .hlist td ul,.mw-parser-output .navbox td.hlist dl,.mw-parser-output .navbox td.hlist ol,.mw-parser-output .navbox td.hlist ul{padding:0.125em 0}.mw-parser-output .navbox .navbar{display:block;font-size:100%}.mw-parser-output .navbox-title .navbar{float:left;text-align:left;margin-right:0.5em}body.skin--responsive .mw-parser-output .navbox-image img{max-width:none!important}@media print{body.ns-0 .mw-parser-output .navbox{display:none!important}}
/* end https://en.wikipedia.org/ */
</style></div><div role="navigation" class="navbox" aria-labelledby="Natural_language_processing454" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Natural_language_processing454" style="font-size:114%;margin:0 4em"><a href="Natural_language_processing" title="Natural language processing">Natural language processing</a></div></th></tr><tr><th scope="row" class="navbox-group" style="width:1%">General terms</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="AI-complete" title="AI-complete">AI-complete</a></li>
<li><a href="Bag-of-words_model" title="Bag-of-words model">Bag-of-words</a></li>
<li><a href="N-gram" title="N-gram"><i>n</i>-gram</a>
<ul><li><a href="Bigram" title="Bigram">Bigram</a></li>
<li><a href="Trigram" title="Trigram">Trigram</a></li></ul></li>
<li><a href="Computational_linguistics" title="Computational linguistics">Computational linguistics</a></li>
<li><a href="Natural_language_understanding" title="Natural language understanding">Natural language understanding</a></li>
<li><a href="Stop_word" title="Stop word">Stop words</a></li>
<li><a href="Text_processing" title="Text processing">Text processing</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Text_mining" title="Text mining">Text analysis</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Argument_mining" title="Argument mining">Argument mining</a></li>
<li><a href="Collocation_extraction" title="Collocation extraction">Collocation extraction</a></li>
<li><a href="Concept_mining" title="Concept mining">Concept mining</a></li>
<li><a href="Coreference#Coreference_resolution" title="Coreference">Coreference resolution</a></li>
<li><a href="Deep_linguistic_processing" title="Deep linguistic processing">Deep linguistic processing</a></li>
<li><a href="Distant_reading" title="Distant reading">Distant reading</a></li>
<li><a href="Information_extraction" title="Information extraction">Information extraction</a></li>
<li><a href="Named-entity_recognition" title="Named-entity recognition">Named-entity recognition</a></li>
<li><a href="Ontology_learning" title="Ontology learning">Ontology learning</a></li>
<li><a href="Parsing" title="Parsing">Parsing</a>
<ul><li><a href="Semantic_parsing" title="Semantic parsing">Semantic parsing</a></li>
<li><a href="Syntactic_parsing_(computational_linguistics)" title="Syntactic parsing (computational linguistics)">Syntactic parsing</a></li></ul></li>
<li><a href="Part-of-speech_tagging" title="Part-of-speech tagging">Part-of-speech tagging</a></li>
<li><a href="Semantic_analysis_(machine_learning)" title="Semantic analysis (machine learning)">Semantic analysis</a></li>
<li><a href="Semantic_role_labeling" title="Semantic role labeling">Semantic role labeling</a></li>
<li><a href="Semantic_decomposition_(natural_language_processing)" title="Semantic decomposition (natural language processing)">Semantic decomposition</a></li>
<li><a href="Semantic_similarity" title="Semantic similarity">Semantic similarity</a></li>
<li><a href="Sentiment_analysis" title="Sentiment analysis">Sentiment analysis</a></li></ul>
<ul><li><a href="Terminology_extraction" title="Terminology extraction">Terminology extraction</a></li>
<li><a href="Text_mining" title="Text mining">Text mining</a></li>
<li><a href="Textual_entailment" title="Textual entailment">Textual entailment</a></li>
<li><a href="Truecasing" title="Truecasing">Truecasing</a></li>
<li><a href="Word-sense_disambiguation" title="Word-sense disambiguation">Word-sense disambiguation</a></li>
<li><a href="Word-sense_induction" title="Word-sense induction">Word-sense induction</a></li></ul>
</div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th id="Text_segmentation21" scope="row" class="navbox-group" style="width:1%"><a href="Text_segmentation" title="Text segmentation">Text segmentation</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Compound-term_processing" title="Compound-term processing">Compound-term processing</a></li>
<li><a href="Lemmatisation" class="mw-redirect" title="Lemmatisation">Lemmatisation</a></li>
<li><a href="Lexical_analysis" title="Lexical analysis">Lexical analysis</a></li>
<li><a href="Shallow_parsing" title="Shallow parsing">Text chunking</a></li>
<li><a href="Stemming" title="Stemming">Stemming</a></li>
<li><a href="Sentence_boundary_disambiguation" title="Sentence boundary disambiguation">Sentence segmentation</a></li>
<li><a href="Word#Word_boundaries" title="Word">Word segmentation</a></li></ul>
</div></td></tr></tbody></table><div>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Automatic_summarization" title="Automatic summarization">Automatic summarization</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Multi-document_summarization" title="Multi-document summarization">Multi-document summarization</a></li>
<li><a href="Sentence_extraction" title="Sentence extraction">Sentence extraction</a></li>
<li><a href="Text_simplification" title="Text simplification">Text simplification</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Machine_translation" title="Machine translation">Machine translation</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Computer-assisted_translation" title="Computer-assisted translation">Computer-assisted</a></li>
<li><a href="Example-based_machine_translation" title="Example-based machine translation">Example-based</a></li>
<li><a href="Rule-based_machine_translation" title="Rule-based machine translation">Rule-based</a></li>
<li><a href="Statistical_machine_translation" title="Statistical machine translation">Statistical</a></li>
<li><a href="Transfer-based_machine_translation" title="Transfer-based machine translation">Transfer-based</a></li>
<li><a href="Neural_machine_translation" title="Neural machine translation">Neural</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Distributional_semantics" title="Distributional semantics">Distributional semantics</a> models</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="BERT_(language_model)" title="BERT (language model)">BERT</a></li>
<li><a href="Document-term_matrix" title="Document-term matrix">Document-term matrix</a></li>
<li><a href="Explicit_semantic_analysis" title="Explicit semantic analysis">Explicit semantic analysis</a></li>
<li><a href="FastText" title="FastText">fastText</a></li>
<li><a href="GloVe" title="GloVe">GloVe</a></li>
<li><a href="Language_model" title="Language model">Language model</a> (<a href="Large_language_model" title="Large language model">large</a>)</li>
<li><a href="Latent_semantic_analysis" title="Latent semantic analysis">Latent semantic analysis</a></li>
<li><a href="Seq2seq" title="Seq2seq">Seq2seq</a></li>
<li><a href="Word2vec" title="Word2vec">Word2vec</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Language_resource" title="Language resource">Language resources</a>,<br>datasets and corpora</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Types and<br>standards</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Corpus_linguistics" title="Corpus linguistics">Corpus linguistics</a></li>
<li><a href="Lexical_resource" title="Lexical resource">Lexical resource</a></li>
<li><a href="Linguistic_Linked_Open_Data" title="Linguistic Linked Open Data">Linguistic Linked Open Data</a></li>
<li><a href="Machine-readable_dictionary" title="Machine-readable dictionary">Machine-readable dictionary</a></li>
<li><a href="Parallel_text" title="Parallel text">Parallel text</a></li>
<li><a href="PropBank" title="PropBank">PropBank</a></li>
<li><a href="Semantic_network" title="Semantic network">Semantic network</a></li>
<li><a href="Simple_Knowledge_Organization_System" title="Simple Knowledge Organization System">Simple Knowledge Organization System</a></li>
<li><a href="Speech_corpus" title="Speech corpus">Speech corpus</a></li>
<li><a href="Text_corpus" title="Text corpus">Text corpus</a></li>
<li><a href="Thesaurus_(information_retrieval)" title="Thesaurus (information retrieval)">Thesaurus (information retrieval)</a></li>
<li><a href="Treebank" title="Treebank">Treebank</a></li>
<li><a href="Universal_Dependencies" title="Universal Dependencies">Universal Dependencies</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Data</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="BabelNet" title="BabelNet">BabelNet</a></li>
<li><a href="Bank_of_English" title="Bank of English">Bank of English</a></li>
<li><a href="DBpedia" title="DBpedia">DBpedia</a></li>
<li><a href="FrameNet" title="FrameNet">FrameNet</a></li>
<li><a href="Google_Ngram_Viewer" class="mw-redirect" title="Google Ngram Viewer">Google Ngram Viewer</a></li>
<li><a href="UBY" title="UBY">UBY</a></li>
<li><a href="WordNet" title="WordNet">WordNet</a></li>
<li><a href="Wikidata" title="Wikidata">Wikidata</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Automatic_identification_and_data_capture" title="Automatic identification and data capture">Automatic identification<br>and data capture</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Speech_recognition" title="Speech recognition">Speech recognition</a></li>
<li><a href="Speech_segmentation" title="Speech segmentation">Speech segmentation</a></li>
<li><a href="Speech_synthesis" title="Speech synthesis">Speech synthesis</a></li>
<li><a href="Natural_language_generation" title="Natural language generation">Natural language generation</a></li>
<li><a href="Optical_character_recognition" title="Optical character recognition">Optical character recognition</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Topic_model" title="Topic model">Topic model</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Document_classification" title="Document classification">Document classification</a></li>
<li><a href="Latent_Dirichlet_allocation" title="Latent Dirichlet allocation">Latent Dirichlet allocation</a></li>
<li><a href="Pachinko_allocation" title="Pachinko allocation">Pachinko allocation</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Computer-assisted_reviewing" title="Computer-assisted reviewing">Computer-assisted<br>reviewing</a></th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Automated_essay_scoring" title="Automated essay scoring">Automated essay scoring</a></li>
<li><a href="Concordancer" title="Concordancer">Concordancer</a></li>
<li><a href="Grammar_checker" title="Grammar checker">Grammar checker</a></li>
<li><a href="Predictive_text" title="Predictive text">Predictive text</a></li>
<li><a href="Pronunciation_assessment" title="Pronunciation assessment">Pronunciation assessment</a></li>
<li><a href="Spell_checker" title="Spell checker">Spell checker</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%"><a href="Natural-language_user_interface" title="Natural-language user interface">Natural language<br>user interface</a></th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Chatbot" title="Chatbot">Chatbot</a></li>
<li><a href="Interactive_fiction" title="Interactive fiction">Interactive fiction</a></li>
<li><a href="Question_answering" title="Question answering">Question answering</a></li>
<li><a href="Virtual_assistant" title="Virtual assistant">Virtual assistant</a></li>
<li><a href="Voice_user_interface" title="Voice user interface">Voice user interface</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Related</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Formal_semantics_(natural_language)" title="Formal semantics (natural language)">Formal semantics</a></li>
<li><a href="Hallucination_(artificial_intelligence)" title="Hallucination (artificial intelligence)">Hallucination</a></li>
<li><a href="Natural_Language_Toolkit" title="Natural Language Toolkit">Natural Language Toolkit</a></li>
<li><a href="SpaCy" title="SpaCy">spaCy</a></li></ul>
</div></td></tr></tbody></table></div>
<div class="navbox-styles"></div><div role="navigation" class="navbox" aria-labelledby="Artificial_intelligence_(AI)426" style="padding:3px"><table class="nowraplinks hlist mw-collapsible autocollapse navbox-inner" style="border-spacing:0;background:transparent;color:inherit"><tbody><tr><th scope="col" class="navbox-title" colspan="2"><div id="Artificial_intelligence_(AI)426" style="font-size:114%;margin:0 4em"><a href="Artificial_intelligence" title="Artificial intelligence">Artificial intelligence</a> (AI)</div></th></tr><tr><td class="navbox-abovebelow" colspan="2"><div>
<ul><li><a href="History_of_artificial_intelligence" title="History of artificial intelligence">History</a>
<ul><li><a href="Timeline_of_artificial_intelligence" title="Timeline of artificial intelligence">timeline</a></li></ul></li>
<li><a href="List_of_artificial_intelligence_companies" title="List of artificial intelligence companies">Companies</a></li>
<li><a href="List_of_artificial_intelligence_projects" title="List of artificial intelligence projects">Projects</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Concepts</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Parameter" title="Parameter">Parameter</a>
<ul><li><a href="Hyperparameter_(machine_learning)" title="Hyperparameter (machine learning)">Hyperparameter</a></li></ul></li>
<li><a href="Loss_functions_for_classification" title="Loss functions for classification">Loss functions</a></li>
<li><a href="Regression_analysis" title="Regression analysis">Regression</a>
<ul><li><a href="Bias%E2%80%93variance_tradeoff" title="Bias–variance tradeoff">Bias–variance tradeoff</a></li>
<li><a href="Double_descent" title="Double descent">Double descent</a></li>
<li><a href="Overfitting" title="Overfitting">Overfitting</a></li></ul></li>
<li><a href="Cluster_analysis" title="Cluster analysis">Clustering</a></li>
<li><a href="Gradient_descent" title="Gradient descent">Gradient descent</a>
<ul><li><a href="Stochastic_gradient_descent" title="Stochastic gradient descent">SGD</a></li>
<li><a href="Quasi-Newton_method" title="Quasi-Newton method">Quasi-Newton method</a></li>
<li><a href="Conjugate_gradient_method" title="Conjugate gradient method">Conjugate gradient method</a></li></ul></li>
<li><a href="Backpropagation" title="Backpropagation">Backpropagation</a></li>
<li><a href="Attention_(machine_learning)" title="Attention (machine learning)">Attention</a></li>
<li><a href="Convolution" title="Convolution">Convolution</a></li>
<li><a href="Normalization_(machine_learning)" title="Normalization (machine learning)">Normalization</a>
<ul><li><a href="Batch_normalization" title="Batch normalization">Batchnorm</a></li></ul></li>
<li><a href="Activation_function" title="Activation function">Activation</a>
<ul><li><a href="Softmax_function" title="Softmax function">Softmax</a></li>
<li><a href="Sigmoid_function" title="Sigmoid function">Sigmoid</a></li>
<li><a href="Rectifier_(neural_networks)" title="Rectifier (neural networks)">Rectifier</a></li></ul></li>
<li><a href="Gating_mechanism" title="Gating mechanism">Gating</a></li>
<li><a href="Weight_initialization" title="Weight initialization">Weight initialization</a></li>
<li><a href="Regularization_(mathematics)" title="Regularization (mathematics)">Regularization</a></li>
<li><a href="Training%2C_validation%2C_and_test_data_sets" title="Training, validation, and test data sets">Datasets</a>
<ul><li><a href="Data_augmentation" title="Data augmentation">Augmentation</a></li></ul></li>
<li><a href="Prompt_engineering" title="Prompt engineering">Prompt engineering</a></li>
<li><a href="Reinforcement_learning" title="Reinforcement learning">Reinforcement learning</a>
<ul><li><a href="Q-learning" title="Q-learning">Q-learning</a></li>
<li><a href="State%E2%80%93action%E2%80%93reward%E2%80%93state%E2%80%93action" title="State–action–reward–state–action">SARSA</a></li>
<li><a href="Imitation_learning" title="Imitation learning">Imitation</a></li>
<li><a href="Policy_gradient_method" title="Policy gradient method">Policy gradient</a></li></ul></li>
<li><a href="Diffusion_process" title="Diffusion process">Diffusion</a></li>
<li><a href="Latent_diffusion_model" title="Latent diffusion model">Latent diffusion model</a></li>
<li><a href="Autoregressive_model" title="Autoregressive model">Autoregression</a></li>
<li><a href="Adversarial_machine_learning" title="Adversarial machine learning">Adversary</a></li>
<li><a href="Retrieval-augmented_generation" title="Retrieval-augmented generation">RAG</a></li>
<li><a href="Uncanny_valley" title="Uncanny valley">Uncanny valley</a></li>
<li><a href="Reinforcement_learning_from_human_feedback" title="Reinforcement learning from human feedback">RLHF</a></li>
<li><a href="Self-supervised_learning" title="Self-supervised learning">Self-supervised learning</a></li>
<li><a href="Reflection_(artificial_intelligence)" class="mw-redirect" title="Reflection (artificial intelligence)">Reflection</a></li>
<li><a href="Recursive_self-improvement" title="Recursive self-improvement">Recursive self-improvement</a></li>
<li><a href="Hallucination_(artificial_intelligence)" title="Hallucination (artificial intelligence)">Hallucination</a></li>
<li><a href="Vibe_coding" title="Vibe coding">Vibe coding</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Applications</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Machine_learning" title="Machine learning">Machine learning</a>
<ul><li><a href="Prompt_engineering#In-context_learning" title="Prompt engineering">In-context learning</a></li></ul></li>
<li><a href="Neural_network_(machine_learning)" title="Neural network (machine learning)">Artificial neural network</a>
<ul><li><a href="Deep_learning" title="Deep learning">Deep learning</a></li></ul></li>
<li><a href="Language_model" title="Language model">Language model</a>
<ul><li><a href="Large_language_model" title="Large language model">Large language model</a></li>
<li><a href="Neural_machine_translation" title="Neural machine translation">NMT</a></li></ul></li>
<li><a href="Reasoning_language_model" title="Reasoning language model">Reasoning language model</a></li>
<li><a href="Model_Context_Protocol" title="Model Context Protocol">Model Context Protocol</a></li>
<li><a href="Intelligent_agent" title="Intelligent agent">Intelligent agent</a></li>
<li><a href="Artificial_human_companion" title="Artificial human companion">Artificial human companion</a></li>
<li><a href="Humanity's_Last_Exam" title="Humanity's Last Exam">Humanity's Last Exam</a></li>
<li><a href="Artificial_general_intelligence" title="Artificial general intelligence">Artificial general intelligence (AGI)</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Implementations</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em"></div><table class="nowraplinks navbox-subgroup" style="border-spacing:0"><tbody><tr><th scope="row" class="navbox-group" style="width:1%">Audio–visual</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="AlexNet" title="AlexNet">AlexNet</a></li>
<li><a href="WaveNet" title="WaveNet">WaveNet</a></li>
<li><a href="Human_image_synthesis" title="Human image synthesis">Human image synthesis</a></li>
<li><a href="Handwriting_recognition" title="Handwriting recognition">HWR</a></li>
<li><a href="Optical_character_recognition" title="Optical character recognition">OCR</a></li>
<li><a href="Computer_vision" title="Computer vision">Computer vision</a></li>
<li><a href="Deep_learning_speech_synthesis" title="Deep learning speech synthesis">Speech synthesis</a>
<ul><li><a href="15.ai" title="15.ai">15.ai</a></li>
<li><a href="ElevenLabs" title="ElevenLabs">ElevenLabs</a></li></ul></li>
<li><a href="Speech_recognition" title="Speech recognition">Speech recognition</a>
<ul><li><a href="Whisper_(speech_recognition_system)" title="Whisper (speech recognition system)">Whisper</a></li></ul></li>
<li><a href="Facial_recognition_system" title="Facial recognition system">Facial recognition</a></li>
<li><a href="AlphaFold" title="AlphaFold">AlphaFold</a></li>
<li><a href="Text-to-image_model" title="Text-to-image model">Text-to-image models</a>
<ul><li><a href="Aurora_(text-to-image_model)" class="mw-redirect" title="Aurora (text-to-image model)">Aurora</a></li>
<li><a href="DALL-E" title="DALL-E">DALL-E</a></li>
<li><a href="Adobe_Firefly" title="Adobe Firefly">Firefly</a></li>
<li><a href="Flux_(text-to-image_model)" title="Flux (text-to-image model)">Flux</a></li>
<li><a href="Ideogram_(text-to-image_model)" title="Ideogram (text-to-image model)">Ideogram</a></li>
<li><a href="Imagen_(text-to-image_model)" title="Imagen (text-to-image model)">Imagen</a></li>
<li><a href="Midjourney" title="Midjourney">Midjourney</a></li>
<li><a href="Recraft" title="Recraft">Recraft</a></li>
<li><a href="Stable_Diffusion" title="Stable Diffusion">Stable Diffusion</a></li></ul></li>
<li><a href="Text-to-video_model" title="Text-to-video model">Text-to-video models</a>
<ul><li><a href="Dream_Machine_(text-to-video_model)" title="Dream Machine (text-to-video model)">Dream Machine</a></li>
<li><a href="Runway_(company)#Services_and_technologies" title="Runway (company)">Runway Gen</a></li>
<li><a href="MiniMax_(company)#Hailuo_AI" title="MiniMax (company)">Hailuo AI</a></li>
<li><a href="Kling_(text-to-video_model)" class="mw-redirect" title="Kling (text-to-video model)">Kling</a></li>
<li><a href="Sora_(text-to-video_model)" title="Sora (text-to-video model)">Sora</a></li>
<li><a href="Veo_(text-to-video_model)" title="Veo (text-to-video model)">Veo</a></li></ul></li>
<li><a href="Music_and_artificial_intelligence" title="Music and artificial intelligence">Music generation</a>
<ul><li><a href="Riffusion" title="Riffusion">Riffusion</a></li>
<li><a href="Suno_AI" title="Suno AI">Suno AI</a></li>
<li><a href="Udio" title="Udio">Udio</a></li></ul></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Text</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Word2vec" title="Word2vec">Word2vec</a></li>
<li><a href="Seq2seq" title="Seq2seq">Seq2seq</a></li>
<li><a href="GloVe" title="GloVe">GloVe</a></li>
<li><a href="BERT_(language_model)" title="BERT (language model)">BERT</a></li>
<li><a href="T5_(language_model)" title="T5 (language model)">T5</a></li>
<li><a href="Llama_(language_model)" title="Llama (language model)">Llama</a></li>
<li><a href="Chinchilla_(language_model)" title="Chinchilla (language model)">Chinchilla AI</a></li>
<li><a href="PaLM" title="PaLM">PaLM</a></li>
<li><a href="Generative_pre-trained_transformer" title="Generative pre-trained transformer">GPT</a>
<ul><li><a href="GPT-1" title="GPT-1">1</a></li>
<li><a href="GPT-2" title="GPT-2">2</a></li>
<li><a href="GPT-3" title="GPT-3">3</a></li>
<li><a href="GPT-J" title="GPT-J">J</a></li>
<li><a href="ChatGPT" title="ChatGPT">ChatGPT</a></li>
<li><a href="GPT-4" title="GPT-4">4</a></li>
<li><a href="GPT-4o" title="GPT-4o">4o</a></li>
<li><a href="OpenAI_o1" title="OpenAI o1">o1</a></li>
<li><a href="OpenAI_o3" title="OpenAI o3">o3</a></li>
<li><a href="GPT-4.5" title="GPT-4.5">4.5</a></li>
<li><a href="GPT-4.1" title="GPT-4.1">4.1</a></li>
<li><a href="OpenAI_o4-mini" title="OpenAI o4-mini">o4-mini</a></li>
<li><a href="GPT-5" title="GPT-5">5</a></li></ul></li>
<li><a href="Claude_(language_model)" title="Claude (language model)">Claude</a></li>
<li><a href="Gemini_(language_model)" title="Gemini (language model)">Gemini</a>
<ul><li><a href="Gemini_(chatbot)" title="Gemini (chatbot)">chatbot</a></li></ul></li>
<li><a href="Grok_(chatbot)" title="Grok (chatbot)">Grok</a></li>
<li><a href="LaMDA" title="LaMDA">LaMDA</a></li>
<li><a href="BLOOM_(language_model)" title="BLOOM (language model)">BLOOM</a></li>
<li><a href="DBRX" title="DBRX">DBRX</a></li>
<li><a href="Project_Debater" title="Project Debater">Project Debater</a></li>
<li><a href="IBM_Watson" title="IBM Watson">IBM Watson</a></li>
<li><a href="IBM_Watsonx" title="IBM Watsonx">IBM Watsonx</a></li>
<li><a href="IBM_Granite" title="IBM Granite">Granite</a></li>
<li><a href="Huawei_PanGu" title="Huawei PanGu">PanGu-Σ</a></li>
<li><a href="DeepSeek_(chatbot)" title="DeepSeek (chatbot)">DeepSeek</a></li>
<li><a href="Qwen" title="Qwen">Qwen</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Decisional</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="AlphaGo" title="AlphaGo">AlphaGo</a></li>
<li><a href="AlphaZero" title="AlphaZero">AlphaZero</a></li>
<li><a href="OpenAI_Five" title="OpenAI Five">OpenAI Five</a></li>
<li><a href="Self-driving_car" title="Self-driving car">Self-driving car</a></li>
<li><a href="MuZero" title="MuZero">MuZero</a></li>
<li><a href="Action_selection" title="Action selection">Action selection</a>
<ul><li><a href="AutoGPT" title="AutoGPT">AutoGPT</a></li></ul></li>
<li><a href="Robot_control" title="Robot control">Robot control</a></li></ul>
</div></td></tr></tbody></table><div></div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">People</th><td class="navbox-list-with-group navbox-list navbox-even" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Alan_Turing" title="Alan Turing">Alan Turing</a></li>
<li><a href="Warren_Sturgis_McCulloch" title="Warren Sturgis McCulloch">Warren Sturgis McCulloch</a></li>
<li><a href="Walter_Pitts" title="Walter Pitts">Walter Pitts</a></li>
<li><a href="John_von_Neumann" title="John von Neumann">John von Neumann</a></li>
<li><a href="Claude_Shannon" title="Claude Shannon">Claude Shannon</a></li>
<li><a href="Shun'ichi_Amari" title="Shun'ichi Amari">Shun'ichi Amari</a></li>
<li><a href="Kunihiko_Fukushima" title="Kunihiko Fukushima">Kunihiko Fukushima</a></li>
<li><a href="Takeo_Kanade" title="Takeo Kanade">Takeo Kanade</a></li>
<li><a href="Marvin_Minsky" title="Marvin Minsky">Marvin Minsky</a></li>
<li><a href="John_McCarthy_(computer_scientist)" title="John McCarthy (computer scientist)">John McCarthy</a></li>
<li><a href="Nathaniel_Rochester_(computer_scientist)" title="Nathaniel Rochester (computer scientist)">Nathaniel Rochester</a></li>
<li><a href="Allen_Newell" title="Allen Newell">Allen Newell</a></li>
<li><a href="Cliff_Shaw" title="Cliff Shaw">Cliff Shaw</a></li>
<li><a href="Herbert_A._Simon" title="Herbert A. Simon">Herbert A. Simon</a></li>
<li><a href="Oliver_Selfridge" title="Oliver Selfridge">Oliver Selfridge</a></li>
<li><a href="Frank_Rosenblatt" title="Frank Rosenblatt">Frank Rosenblatt</a></li>
<li><a href="Bernard_Widrow" title="Bernard Widrow">Bernard Widrow</a></li>
<li><a href="Joseph_Weizenbaum" title="Joseph Weizenbaum">Joseph Weizenbaum</a></li>
<li><a href="Seymour_Papert" title="Seymour Papert">Seymour Papert</a></li>
<li><a href="Seppo_Linnainmaa" title="Seppo Linnainmaa">Seppo Linnainmaa</a></li>
<li><a href="Paul_Werbos" title="Paul Werbos">Paul Werbos</a></li>
<li><a href="Geoffrey_Hinton" title="Geoffrey Hinton">Geoffrey Hinton</a></li>
<li><a href="John_Hopfield" title="John Hopfield">John Hopfield</a></li>
<li><a href="J%C3%BCrgen_Schmidhuber" title="Jürgen Schmidhuber">Jürgen Schmidhuber</a></li>
<li><a href="Yann_LeCun" title="Yann LeCun">Yann LeCun</a></li>
<li><a href="Yoshua_Bengio" title="Yoshua Bengio">Yoshua Bengio</a></li>
<li><a href="Lotfi_A._Zadeh" title="Lotfi A. Zadeh">Lotfi A. Zadeh</a></li>
<li><a href="Stephen_Grossberg" title="Stephen Grossberg">Stephen Grossberg</a></li>
<li><a href="Alex_Graves_(computer_scientist)" title="Alex Graves (computer scientist)">Alex Graves</a></li>
<li><a href="James_Goodnight" title="James Goodnight">James Goodnight</a></li>
<li><a href="Andrew_Ng" title="Andrew Ng">Andrew Ng</a></li>
<li><a href="Fei-Fei_Li" title="Fei-Fei Li">Fei-Fei Li</a></li>
<li><a href="Ilya_Sutskever" title="Ilya Sutskever">Ilya Sutskever</a></li>
<li><a href="Alex_Krizhevsky" title="Alex Krizhevsky">Alex Krizhevsky</a></li>
<li><a href="Ian_Goodfellow" title="Ian Goodfellow">Ian Goodfellow</a></li>
<li><a href="Demis_Hassabis" title="Demis Hassabis">Demis Hassabis</a></li>
<li><a href="David_Silver_(computer_scientist)" title="David Silver (computer scientist)">David Silver</a></li>
<li><a href="Andrej_Karpathy" title="Andrej Karpathy">Andrej Karpathy</a></li>
<li><a href="Ashish_Vaswani" title="Ashish Vaswani">Ashish Vaswani</a></li>
<li><a href="Noam_Shazeer" title="Noam Shazeer">Noam Shazeer</a></li>
<li><a href="Aidan_Gomez" title="Aidan Gomez">Aidan Gomez</a></li>
<li><a href="Mustafa_Suleyman" title="Mustafa Suleyman">Mustafa Suleyman</a></li>
<li><a href="Fran%C3%A7ois_Chollet" title="François Chollet">François Chollet</a></li></ul>
</div></td></tr><tr><th scope="row" class="navbox-group" style="width:1%">Architectures</th><td class="navbox-list-with-group navbox-list navbox-odd" style="width:100%;padding:0"><div style="padding:0 0.25em">
<ul><li><a href="Neural_Turing_machine" title="Neural Turing machine">Neural Turing machine</a></li>
<li><a href="Differentiable_neural_computer" title="Differentiable neural computer">Differentiable neural computer</a></li>
<li><a href="Transformer_(deep_learning_architecture)" title="Transformer (deep learning architecture)">Transformer</a>
<ul><li><a href="Vision_transformer" title="Vision transformer">Vision transformer (ViT)</a></li></ul></li>
<li><a href="Recurrent_neural_network" title="Recurrent neural network">Recurrent neural network (RNN)</a></li>
<li><a href="Long_short-term_memory" title="Long short-term memory">Long short-term memory (LSTM)</a></li>
<li><a href="Gated_recurrent_unit" title="Gated recurrent unit">Gated recurrent unit (GRU)</a></li>
<li><a href="Echo_state_network" title="Echo state network">Echo state network</a></li>
<li><a href="Multilayer_perceptron" title="Multilayer perceptron">Multilayer perceptron (MLP)</a></li>
<li><a href="Convolutional_neural_network" title="Convolutional neural network">Convolutional neural network (CNN)</a></li>
<li><a href="Residual_neural_network" title="Residual neural network">Residual neural network (RNN)</a></li>
<li><a href="Highway_network" title="Highway network">Highway network</a></li>
<li><a href="Mamba_(deep_learning_architecture)" title="Mamba (deep learning architecture)">Mamba</a></li>
<li><a href="Autoencoder" title="Autoencoder">Autoencoder</a></li>
<li><a href="Variational_autoencoder" title="Variational autoencoder">Variational autoencoder (VAE)</a></li>
<li><a href="Generative_adversarial_network" title="Generative adversarial network">Generative adversarial network (GAN)</a></li>
<li><a href="Graph_neural_network" title="Graph neural network">Graph neural network (GNN)</a></li></ul>
</div></td></tr><tr><td class="navbox-abovebelow" colspan="2"><div>
<ul><li><span class="noviewer" typeof="mw:File"><span title="Category"></span></span> Category</li></ul>
</div></td></tr></tbody></table></div></div><!--htdig_noindex--><div><div class="zim-footer">
This article is issued from <a class="external text" title="Last edited on 2025-07-17" href="https://en.wikipedia.org/wiki/?title=Word_embedding&oldid=1300900105">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.
</div>
</div><!--/htdig_noindex--></div>
</div>
</main>
</div>
</div>
</div>
</body></html>